09:30:12,916 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:12,957 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3019880840516011621] 09:30:12,957 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:12,962 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:12,962 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:12,972 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:12,972 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:12,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:12,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:12,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:12,986 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:12,995 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3132507440257468749] 09:30:12,995 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:13,5 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:13,5 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:13,9 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:13,9 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:13,20 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:13,39 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2241403699834551345] 09:30:13,39 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:13,43 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:13,43 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:13,54 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:13,56 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:13,57 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:13,57 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:13,58 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 09:30:13,69 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#661621000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,70 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 09:30:13,76 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:13,76 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4725dc99 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ba61e33 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,76 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@284e8ed8 09:30:13,80 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1714037413079, None)) 09:30:13,80 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037413078,0,0)) 09:30:13,80 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1d4762a7 09:30:13,80 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3ba61e33 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2305cea 09:30:13,80 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 09:30:13,81 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:13,81 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,81 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:13,83 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:13,85 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 09:30:13,92 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:13,109 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785] 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,113 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781] 09:30:13,113 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1231653833] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,113 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:13,114 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 09:30:13,114 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 09:30:13,115 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 09:30:13,115 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 09:30:13,115 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,115 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,115 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-1124455023 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,115 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#-1124455023] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,121 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 09:30:13,121 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:13,121 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a92c798 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2997dcae 09:30:13,121 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@463e6f88 09:30:13,121 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1714037413121, None)) 09:30:13,121 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 09:30:13,121 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 09:30:13,122 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:13,122 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,122 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,122 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037413120,0,0)) 09:30:13,122 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5d2a1267 09:30:13,122 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2997dcae to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@18c08c0b 09:30:13,122 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1201922360] 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:13,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#736830299] 09:30:13,124 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 09:30:13,124 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#1201922360] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,124 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 09:30:13,124 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#736830299] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,124 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#2100518240 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 09:30:13,124 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1559567625] 09:30:13,125 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#2100518240] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,125 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#-1559567625] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,125 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-216646173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,125 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1675028122] 09:30:13,130 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1373547184 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,130 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1961954474] 09:30:13,134 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#1373547184] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,134 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1961954474] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,135 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1931609429] 09:30:13,135 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1931609429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,136 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 09:30:13,137 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 09:30:13,137 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#-216646173 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,137 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:13,137 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:13,137 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:13,137 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:13,137 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 09:30:13,137 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 09:30:13,138 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:13,138 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:13,139 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#1675028122] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 09:30:13,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 09:30:13,141 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 09:30:13,142 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:30:13,147 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 09:30:13,147 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 09:30:13,147 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,147 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,147 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 09:30:13,147 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:30:13,148 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,148 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,148 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 09:30:13,148 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,148 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:30:13,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,148 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,148 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,148 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 09:30:13,148 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:30:13,148 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 09:30:13,149 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 09:30:13,149 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 09:30:13,149 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 09:30:13,150 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:13,155 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:13,173 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#661621000] 09:30:13,173 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 09:30:13,173 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785] 09:30:13,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:13,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:30:13,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 09:30:13,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,395 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 09:30:13,396 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:13,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 09:30:13,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,396 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 09:30:13,396 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:13,425 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:30:13,425 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 09:30:13,426 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 09:30:13,687 AM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:13,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:30:13,705 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,705 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 09:30:13,724 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:30:13,724 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,725 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 09:30:13,747 AM [cluster-test-shard-dispatcher-32] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:13,975 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:30:13,975 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,975 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 09:30:13,987 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:13,987 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:13,988 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:13,989 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,5 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:30:14,5 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,6 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,18 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:14,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 09:30:14,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:14,27 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:14,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=null 09:30:14,28 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:14,28 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:30:14,30 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:30:14,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:30:14,40 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:30:14,41 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 09:30:14,41 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 09:30:14,42 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:14,42 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 09:30:14,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 09:30:14,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:30:14,55 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 09:30:14,57 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:30:14,57 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:14,57 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:14,62 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:30:14,62 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:14,68 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:14,235 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:30:14,235 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:14,236 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 09:30:14,236 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,241 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:14,241 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 09:30:14,241 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:14,241 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:14,241 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 09:30:14,242 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:14,242 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:14,242 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:14,245 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:14,245 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:14,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 09:30:14,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:14,246 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:14,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:30:14,246 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 09:30:14,248 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 09:30:14,248 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:30:14,248 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:14,248 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:14,248 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 09:30:14,248 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 09:30:14,250 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 09:30:14,251 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 09:30:14,252 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:30:14,252 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:14,280 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 09:30:14,292 AM [cluster-test-shard-dispatcher-25] [WARN] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 223, lastApplied : -1, commitIndex : -1 09:30:14,340 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#661621000] 09:30:14,340 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 09:30:14,340 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781] 09:30:14,341 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,341 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1231653833] 09:30:14,341 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,341 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1231653833] 09:30:14,341 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,341 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,341 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:14,341 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,341 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 09:30:14,342 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 09:30:14,342 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,342 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 09:30:14,342 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:14,342 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=12] 09:30:14,342 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:30:14,342 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 09:30:14,342 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=12] 09:30:14,342 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=12] 09:30:14,342 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=12] 09:30:14,342 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 09:30:14,342 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars 09:30:14,342 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 2 ] 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 09:30:14,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 09:30:14,350 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 09:30:14,350 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, disabling transaction tracking 09:30:14,350 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,350 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,350 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,350 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,351 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:14,351 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:14,353 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:14,353 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,353 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:14,353 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6c890dc7 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 12 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6c367197 09:30:14,360 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,360 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 2 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,361 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 09:30:14,361 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:30:14,361 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 of 2 operations, client version 12 09:30:14,361 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,361 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 ready 09:30:14,361 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 09:30:14,362 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,362 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,362 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,362 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 09:30:14,367 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}]]] 09:30:14,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,367 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:14,367 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:14,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,368 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,368 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,368 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 09:30:14,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 1 09:30:14,368 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 1 09:30:14,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,368 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,368 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] resolved 09:30:14,368 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] resolved 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: successfully resolved all cohort actors 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finishCanCommit 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,369 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:14,370 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,370 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65}] 09:30:14,370 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 09:30:14,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:30:14,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:14,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 09:30:14,376 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=26469805} is already disabled 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:30:14,377 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:14,377 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=26991273} is already disabled 09:30:14,378 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:14,378 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,378 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:14,379 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 09:30:14,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 09:30:14,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:14,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}]]] 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, size=65} 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,382 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:14,382 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:14,382 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,382 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,383 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,383 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:14,383 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67}] 09:30:14,383 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 09:30:14,385 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 09:30:14,385 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:14,386 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:14,386 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 validated 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:14,386 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: success 09:30:14,389 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:14,389 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit returning result: true 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0}, size=67} 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 3 09:30:14,389 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:14,389 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:14,389 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 commit 09:30:14,389 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 finish commit 09:30:14,389 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,389 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,390 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,390 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:14,393 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,393 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,393 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 candidate ready 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: canCommit - messages: [] 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 prepared 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,394 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:14,395 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:14,395 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 09:30:14,396 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 09:30:14,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}]]] 09:30:14,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 submitted to persistence 09:30:14,397 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 09:30:14,398 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=3, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}]]] 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=180} 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:30:14,399 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}]]] 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=176} 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:30:14,399 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,400 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=3, payload=Simple{size=180}] 09:30:14,400 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=176}] 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%231715406785$e] 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 09:30:14,401 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%231928534781$f] 09:30:14,401 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1527867497] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 09:30:14,402 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 09:30:14,402 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1537792691] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 09:30:14,402 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:14,402 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:30:14,402 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 09:30:14,402 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:14,402 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1537792691] 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1527867497] 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 152.5 μs 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1537792691] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#1527867497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,403 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 180.2 μs 09:30:14,407 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0: commit succeeded 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:14,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=180} 09:30:14,408 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data Simple{size=176} 09:30:14,408 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,408 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,408 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:14,408 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:14,408 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 09:30:14,408 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 09:30:14,409 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 09:30:14,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 09:30:14,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:14,409 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 09:30:14,409 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 09:30:14,409 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 09:30:14,409 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 09:30:14,409 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:14,410 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-476272093] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 09:30:14,410 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-476272093] 09:30:14,410 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 3 09:30:14,411 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,411 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:30:14,411 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 74.22 μs 09:30:14,411 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-476272093] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,412 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-724071075] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 09:30:14,413 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-724071075] 09:30:14,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 09:30:14,413 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,413 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-724071075] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,413 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:30:14,413 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 73.36 μs 09:30:14,415 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,415 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,417 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,417 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@31bc4e6a 09:30:14,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3a665e16 09:30:14,424 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738] 09:30:14,424 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,425 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_2#-1205478075] 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738)] 09:30:14,427 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_2#-1205478075, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] 09:30:14,428 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738)] 09:30:14,428 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 09:30:14,440 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 09:30:14,440 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 09:30:14,441 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 09:30:14,441 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: false 09:30:14,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_2#-1205478075)] 09:30:14,441 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_2#-1205478075)] 09:30:14,451 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with BatchedModificationsReply [numBatched=1] 09:30:14,451 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ReadData operation succeeded 09:30:14,451 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying 2 components for commit 09:30:14,451 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 09:30:14,451 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 09:30:14,451 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 09:30:14,451 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 09:30:14,451 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_1#1018600738)] 09:30:14,451 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 Readying transaction for shard people 09:30:14,451 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 = [ 1 ] 09:30:14,451 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction called 09:30:14,451 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 sending 0 batched modifications, ready: true 09:30:14,451 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:1-1_2#-1205478075)] 09:30:14,452 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,452 AM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,452 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 09:30:14,452 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying 1 components for commit 09:30:14,452 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Readying transaction for shard people 09:30:14,452 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 09:30:14,452 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 canCommit 09:30:14,454 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 12 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, client version 12 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 ready 09:30:14,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] resolved 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=12] 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 readyTransaction 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] resolved 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: successfully resolved all cohort actors 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finishCanCommit 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 modifications completed with ReadyTransactionReply [version=12] 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 - previous Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 readied - proceeding to FindPrimaryShard 09:30:14,458 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people 09:30:14,459 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:30:14,459 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 09:30:14,459 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 directCommit called 09:30:14,459 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 sending 1 batched modifications, ready: true 09:30:14,459 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Readying Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 of 1 operations, client version 12 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 ready 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 09:30:14,462 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 is scheduled for canCommit step 09:30:14,463 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:14,463 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,463 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Can committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 validated 09:30:14,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:14,464 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: success 09:30:14,465 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:14,465 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit returning result: true 09:30:14,465 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 commit 09:30:14,465 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 finish commit 09:30:14,465 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,465 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1715406785)] 09:30:14,465 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,465 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#1928534781)] 09:30:14,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 235 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:30:14,468 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}]]] 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 candidate ready 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: canCommit - messages: [] 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 prepared 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 224 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 validated 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: canCommit for member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: success 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Committing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,469 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 candidate ready 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: canCommit - messages: [] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 prepared 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Finishing commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 scheduled for commit step 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 submitted to persistence 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 221 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 09:30:14,470 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 submitted to persistence 09:30:14,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 09:30:14,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 09:30:14,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}]]] 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=235} 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:30:14,473 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,474 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=235}] 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=3, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}], SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}]]] 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=224} 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=221} 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 09:30:14,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:30:14,475 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-cars-testTransactionChainWithMultipleShards%231715406785$r] 09:30:14,475 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,476 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 144.5 μs 09:30:14,476 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 09:30:14,476 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:30:14,476 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 09:30:14,476 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=3, payload=Simple{size=224}] 09:30:14,476 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=3, payload=Simple{size=221}] 09:30:14,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 09:30:14,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:30:14,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 09:30:14,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:30:14,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=235} 09:30:14,480 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,481 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,481 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 75.78 μs 09:30:14,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 09:30:14,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 09:30:14,482 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards%231928534781$s] 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 09:30:14,483 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithMultipleShards_member-1-shard-people-testTransactionChainWithMultipleShards$p] 09:30:14,483 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 45.60 μs 09:30:14,483 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 09:30:14,483 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 23.93 μs 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:30:14,483 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 09:30:14,485 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0: commit succeeded 09:30:14,485 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=12] 09:30:14,485 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 canCommit 09:30:14,485 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 successfully completed direct commit 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=224} 09:30:14,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:30:14,486 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=221} 09:30:14,486 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 41.54 μs 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:30:14,486 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 3 09:30:14,486 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 26.28 μs 09:30:14,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 09:30:14,486 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:30:14,486 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:14,486 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:14,486 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] found - trying create transaction 09:30:14,486 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards)] 09:30:14,487 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:30:14,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:14,490 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:30:14,491 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_3#-1788510782] 09:30:14,492 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_3#-1788510782, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=12] 09:30:14,492 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:14,492 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_3#-1788510782)] 09:30:14,497 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 09:30:14,498 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 09:30:14,498 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:14,498 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] found - trying create transaction 09:30:14,498 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards)] 09:30:14,498 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:30:14,499 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Creating transaction : member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:30:14,499 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_4#915135677] 09:30:14,500 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_4#915135677, transactionId=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0, version=12] 09:30:14,501 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] 09:30:14,501 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_4#915135677)] 09:30:14,503 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 ReadData operation succeeded 09:30:14,503 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,503 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,503 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,503 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 09:30:14,503 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onShutDown 09:30:14,503 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onShutDown 09:30:14,504 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 09:30:14,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 09:30:14,504 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:30:14,504 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 09:30:14,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:30:14,504 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:30:14,504 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:30:14,504 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 09:30:14,505 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:30:14,505 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:30:14,505 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 09:30:14,505 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:30:14,505 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:14,505 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 09:30:14,505 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 09:30:14,505 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 09:30:14,506 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1231653833] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#1231653833] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:14,506 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 09:30:14,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 09:30:14,506 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:14,508 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:14,509 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@647c6370 09:30:14,509 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78b5cd61 09:30:14,509 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@647c6370 09:30:14,509 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78b5cd61 09:30:14,510 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 09:30:14,510 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 09:30:14,510 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:30:14,510 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:30:14,735 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionChainWithMultipleShards: Failed to transfer leadership in 226.5 ms 09:30:14,735 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In unpauseLeader 09:30:14,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader transfer failed - sending PoisonPill 09:30:14,738 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leadership transfer expired 09:30:14,740 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:30:14,740 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-631: Closing 09:30:14,740 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:30:14,755 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testTransactionChainWithMultipleShards: Failed to transfer leadership in 246.5 ms 09:30:14,755 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In unpauseLeader 09:30:14,755 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader transfer failed - sending PoisonPill 09:30:14,755 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leadership transfer expired 09:30:14,756 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 09:30:14,756 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-630: Closing 09:30:14,756 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:30:14,756 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:30:14,756 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 09:30:14,756 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:30:14,757 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 09:30:14,757 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:30:14,759 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:14,759 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:14,773 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:14,773 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:14,773 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:14,773 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:14,776 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:14,787 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:14,788 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:15,779 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3019880840516011621] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:15,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3132507440257468749] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:15,811 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:15,811 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:15,813 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:15.815] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:15,829 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:15,830 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:15,838 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:15.839] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:15,857 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:30:15,866 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5934265762357882629] 09:30:15,867 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:15,868 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:15,868 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:15,876 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:15,876 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:15,884 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:15,891 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [8692731048900739734] 09:30:15,892 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:15,894 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:15,894 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:15,904 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:15,904 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:15,928 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:15,939 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-173605899490485357] 09:30:15,939 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:15,941 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:15,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:15,948 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:15,948 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:15,952 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:15,954 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:15,956 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 09:30:15,980 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-530962153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:15,981 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 09:30:15,982 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 09:30:15,982 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 09:30:15,982 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 09:30:15,982 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 09:30:15,982 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@21b0648b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4adeacfc 09:30:15,982 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:15,982 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:15,995 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1e6f0aad 09:30:15,995 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1714037415983, None)) 09:30:15,996 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037415982,0,0)) 09:30:15,996 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4adeacfc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3e4e7df9 09:30:15,996 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7f31356e 09:30:15,996 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 09:30:15,997 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:16,0 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:16,2 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 09:30:16,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:16,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 09:30:16,8 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:16,8 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 09:30:16,8 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,8 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,8 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228] 09:30:16,8 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 09:30:16,9 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 09:30:16,9 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,9 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#982942240] 09:30:16,9 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#482462331 created and ready for shard:member-1-shard-cars-testInstallSnapshot 09:30:16,9 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#482462331] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,10 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#982942240] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#2054024886] 09:30:16,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 09:30:16,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 09:30:16,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 09:30:16,10 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 09:30:16,11 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 09:30:16,12 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#161806395] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 09:30:16,12 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 1.072 ms 09:30:16,12 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 1.892 ms: snapshotIndex=5, snapshotTerm=1, journal-size=0 09:30:16,12 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#161806395] 09:30:16,12 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 09:30:16,12 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#161806395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,13 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 229.3 μs 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 3.824 ms - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 09:30:16,14 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 09:30:16,14 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 09:30:16,15 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 09:30:16,19 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1517022354] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,19 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:16,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 09:30:16,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 09:30:16,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@56bc5c47 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6efa457e 09:30:16,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2bf8c6c5 09:30:16,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 09:30:16,22 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 09:30:16,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:16,22 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,23 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1714037416023, None)) 09:30:16,23 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037416022,0,0)) 09:30:16,23 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7842249c 09:30:16,24 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6efa457e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@34f38229 09:30:16,24 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 09:30:16,24 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 09:30:16,24 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:16,24 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 09:30:16,26 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#1907434473] 09:30:16,27 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 09:30:16,28 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 09:30:16,28 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#1907434473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,28 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-2063013531] 09:30:16,28 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1209683430 created and ready for shard:member-2-shard-cars-testInstallSnapshot 09:30:16,28 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1209683430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,28 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#-2063013531] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 09:30:16,30 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 09:30:16,31 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 09:30:16,31 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 09:30:16,32 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:16,35 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 09:30:16,39 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:16,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-530962153] 09:30:16,75 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 09:30:16,75 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228] 09:30:16,264 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:16,264 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 09:30:16,265 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 09:30:16,265 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 09:30:16,484 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:30:16,484 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:16,485 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 09:30:16,552 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:16,724 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:30:16,724 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:16,724 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 09:30:16,886 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:16,886 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,886 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:16,887 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,904 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:30:16,959 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:17,35 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:30:17,35 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:17,35 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 09:30:17,35 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:17,40 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 09:30:17,44 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 09:30:17,44 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=12] 09:30:17,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:17,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:17,45 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 09:30:17,45 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 09:30:17,46 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 09:30:17,46 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:30:17,47 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=12] 09:30:17,47 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:17,47 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:17,47 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 09:30:17,47 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 09:30:17,48 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 09:30:17,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:30:17,51 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 09:30:17,51 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@2dc974bf, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 09:30:17,51 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 09:30:17,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 09:30:17,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 09:30:17,53 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 09:30:17,53 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 09:30:17,53 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 09:30:17,53 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 09:30:17,54 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 09:30:17,56 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 09:30:17,57 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 09:30:17,64 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:30:17,66 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:30:17,66 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 09:30:17,67 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@668f9602, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 09:30:17,67 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 09:30:17,69 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 09:30:17,69 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-184393202] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 326.7 μs 09:30:17,70 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 09:30:17,70 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-184393202] 09:30:17,70 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 09:30:17,71 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 190.7 μs 09:30:17,71 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-184393202] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 09:30:17,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 09:30:17,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 09:30:17,72 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 09:30:17,77 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 09:30:17,77 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 09:30:17,81 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 09:30:17,81 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-530962153] 09:30:17,81 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 Queuing TransactionOperation 09:30:17,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:17,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@741eea7d] 09:30:17,82 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@741eea7d] 09:30:17,82 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228)] for shard cars 09:30:17,82 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:17,82 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@512fe325 09:30:17,83 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 09:30:17,83 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-2-shard-cars-testInstallSnapshot 09:30:17,84 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 09:30:17,84 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onShutDown 09:30:17,84 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 09:30:17,84 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 09:30:17,84 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 09:30:17,84 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 09:30:17,84 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 09:30:17,85 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 09:30:17,85 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 09:30:17,85 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: shutting down 09:30:17,86 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 09:30:17,86 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1517022354] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1517022354] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,86 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1517022354] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#-1517022354] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,86 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-561496974] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-561496974] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 09:30:17,86 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 09:30:17,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 09:30:17,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 09:30:17,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@222b76b2 09:30:17,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@222b76b2 09:30:17,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 09:30:17,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:30:17,88 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=12] 09:30:17,89 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:17,89 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:17,90 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,90 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#-626796228] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:17,394 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testInstallSnapshot: Failed to transfer leadership in 307.4 ms 09:30:17,394 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In unpauseLeader 09:30:17,394 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: leader transfer failed - sending PoisonPill 09:30:17,394 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leadership transfer expired 09:30:17,395 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 09:30:17,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-636: Closing 09:30:17,395 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 09:30:17,396 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 09:30:17,396 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 09:30:17,396 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 09:30:17,397 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 09:30:17,399 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:17,399 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:17,411 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:17,411 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:17,411 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:17,411 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:17,413 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:17,426 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:17,426 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:18,419 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5934265762357882629] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:18,445 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:18,445 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:18,447 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:18.449] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:18,449 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [8692731048900739734] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:18,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:18,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:18,470 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:18.472] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:18,490 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:18,499 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1668899775322469482] 09:30:18,499 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:18,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:18,501 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:18,509 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:18,509 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:18,514 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:18,514 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:18,514 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:18,517 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:18,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4181752327800989664] 09:30:18,531 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:18,543 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:18,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:18,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:18,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:18,554 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 09:30:18,554 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3a836e4b aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} 09:30:18,554 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-people-member-2:datastore-testTransactionChainWithMultipleShards@0:0_4#915135677)] 09:30:18,554 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2070214283]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@5f84c0a4 09:30:18,554 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-1-testWriteTransactionWithMultipleShards#696398894], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars-1, dataTree=present}} 09:30:18,554 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 closeTransaction called 09:30:18,554 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1f9d64ff transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2070214283]}, enqueuedTicks=624848105134} 09:30:18,554 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-cars-member-2:datastore-testTransactionChainWithMultipleShards@0:0_3#-1788510782)] 09:30:18,554 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1b919b3f aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} 09:30:18,554 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2070214283]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@3a239f1 09:30:18,555 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-1-testWriteTransactionWithMultipleShards#-2069477575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people-1, dataTree=present}} 09:30:18,555 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3bc82868 transmitting entry ConnectionEntry{request=AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#2070214283]}, enqueuedTicks=624848300528} 09:30:18,555 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-1-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-3-0} 09:30:18,605 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:18,612 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:18,615 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:18,638 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2185439435651038318] 09:30:18,639 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:18,641 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:18,641 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:18,648 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:18,648 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:18,653 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:18,655 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:18,657 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:30:18,663 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-787409275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,663 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:18,666 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:30:18,666 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:18,666 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@103ea72a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b6aec13 09:30:18,666 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5e68611a 09:30:18,667 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:18,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,667 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:18,667 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,667 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037418667, None)) 09:30:18,668 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037418666,0,0)) 09:30:18,668 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b6aec13 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6f7c3811 09:30:18,668 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4bc7b56 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,669 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:18,670 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:30:18,671 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:18,672 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:30:18,672 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,672 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1659926845] 09:30:18,673 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:30:18,674 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#1659926845] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 09:30:18,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 09:30:18,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1249572135 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 09:30:18,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 09:30:18,679 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1249572135] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,679 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,679 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:18,680 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 09:30:18,683 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:30:18,683 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:18,684 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@c729838 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d1f4608 09:30:18,687 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2c4a3987 09:30:18,687 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037418684, None)) 09:30:18,688 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:18,688 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037418683,0,0)) 09:30:18,688 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,688 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d1f4608 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@331a7ddd 09:30:18,688 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:18,688 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,688 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@512461e2 09:30:18,688 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:30:18,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:18,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,689 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1136523664] 09:30:18,690 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:30:18,691 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1136523664] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,691 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#584560916 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,693 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:30:18,693 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#638403304] 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 09:30:18,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 09:30:18,695 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 09:30:18,695 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 09:30:18,705 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:18,708 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#584560916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,708 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#638403304] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:18,708 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 09:30:18,715 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:18,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-787409275] 09:30:18,739 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:30:18,739 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:18,905 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:30:18,906 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 09:30:18,906 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 09:30:19,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:30:19,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,175 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 09:30:19,234 AM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:19,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:30:19,475 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,475 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 09:30:19,526 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:19,527 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,528 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,542 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,543 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,668 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:19,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:30:19,795 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,796 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 09:30:19,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,796 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:19,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:30:19,805 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 09:30:19,806 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 09:30:19,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=12] 09:30:19,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:19,806 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:19,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 09:30:19,806 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 09:30:19,807 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=12] 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:19,808 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 09:30:19,810 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 09:30:19,810 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:19,850 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:19,853 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:19,856 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:30:19,861 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:19,862 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1316809954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:19,863 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:19,863 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:19,864 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,864 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:30:19,864 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:19,864 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4d6468b1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e2dd173 09:30:19,864 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@766ad29b 09:30:19,868 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037419868, None)) 09:30:19,868 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037419867,0,0)) 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@ab9f428 09:30:19,869 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e2dd173 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1fab129d 09:30:19,869 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,869 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#526513624] 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#526513624] 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#526513624] 09:30:19,870 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:30:19,870 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#526513624] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:19,871 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:19,871 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-128998604 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,871 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-340001213] 09:30:19,871 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:30:19,871 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:19,871 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:30:19,871 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:19,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:19,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:19,882 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:19,882 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:19,882 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:19,882 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:19,883 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:19,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-787409275] 09:30:19,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:30:19,898 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:19,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:19,929 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 09:30:20,547 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:30:20,885 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2185439435651038318] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:20,886 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [-2185439435651038318] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:25,651 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 09:30:25,653 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:25,653 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:25,686 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 09:30:25,686 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:25,843 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] 09:30:25,843 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:30:25,843 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1136523664] 09:30:25,843 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:25,843 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:25,843 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:30:25,844 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:30:25,844 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:30:25,844 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1136523664] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,844 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1136523664] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,844 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,844 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,845 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] 09:30:25,845 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:25,846 AM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:30:25,846 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:25,846 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:25,847 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:25,847 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 09:30:25,847 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 09:30:25,847 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:25,847 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 09:30:25,847 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:30:25,852 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 09:30:25,852 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:25,853 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:25,853 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:30:25,853 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:25,853 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 09:30:25,853 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:25,853 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 09:30:25,853 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 09:30:25,853 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 09:30:25,853 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,853 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1793812040] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1793812040] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f5a6099 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f5a6099 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=12] 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:25,854 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:25,855 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#1279165656] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:25,857 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1431925189] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:26,166 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Failed to transfer leadership in 312.0 ms 09:30:26,166 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In unpauseLeader 09:30:26,166 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: leader transfer failed - sending PoisonPill 09:30:26,166 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leadership transfer expired 09:30:26,167 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:30:26,167 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-641: Closing 09:30:26,167 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:30:26,167 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 09:30:26,167 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:30:26,168 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:26,168 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 09:30:26,168 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:30:26,170 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:26,170 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:27,185 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1668899775322469482] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:27,188 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:27,189 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:27,190 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:27.193] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:27,212 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:27,212 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:27,215 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:27.217] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:27,238 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:30:27,250 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7816564040306612964] 09:30:27,250 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:27,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:27,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:27,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:27,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:27,265 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:27,265 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:27,265 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:27,269 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:27,275 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3105653856253364115] 09:30:27,276 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:27,277 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:27,277 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:27,284 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:27,284 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:27,292 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:27,303 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6610563968208275743] 09:30:27,304 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:27,306 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:27,306 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:27,316 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:27,316 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:27,317 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:27,318 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:27,320 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 09:30:27,326 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:27,329 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1793815522] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 09:30:27,334 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4e01229b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ac258bd 09:30:27,334 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2478bd32 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1714037427334, None)) 09:30:27,334 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:27,334 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,334 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037427333,0,0)) 09:30:27,334 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@581a33fd 09:30:27,335 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5ac258bd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@55054e10 09:30:27,335 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 09:30:27,335 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:27,335 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:27,335 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,337 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:27,339 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 09:30:27,340 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,340 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,340 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954] 09:30:27,341 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 09:30:27,341 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 09:30:27,342 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,342 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1672821583 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,342 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#-1672821583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:27,346 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:27,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:27,346 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 09:30:27,346 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:30:27,346 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 09:30:27,346 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 09:30:27,346 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 09:30:27,346 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 09:30:27,349 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#1197435699] 09:30:27,349 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#1197435699] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,357 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:27,366 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:27,371 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,375 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:27,375 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2210710c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@63b1f9b5 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@26b36ba7 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1714037427379, None)) 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037427378,0,0)) 09:30:27,379 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@63b1f9b5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@175f097a 09:30:27,381 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@46ee1e34 09:30:27,381 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:27,381 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,381 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 09:30:27,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:27,382 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,383 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,383 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] 09:30:27,384 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 09:30:27,384 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,384 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 09:30:27,385 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#840569734] 09:30:27,385 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1499708519 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,385 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1499708519] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,385 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#840569734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:27,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 09:30:27,386 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:30:27,386 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,386 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 09:30:27,386 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 09:30:27,405 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:27,417 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:27,432 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1793815522] 09:30:27,432 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:30:27,432 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954] 09:30:27,643 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:27,643 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:27,644 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:30:27,645 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 09:30:27,645 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 09:30:27,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:30:27,883 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:27,883 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 09:30:27,957 AM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:28,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:30:28,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,183 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,268 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,269 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:28,269 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,277 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,330 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:28,503 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:30:28,503 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,503 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 09:30:28,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 09:30:28,510 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=12] 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 09:30:28,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 09:30:28,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=12] 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:28,512 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 09:30:28,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 09:30:28,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:28,588 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] 09:30:28,589 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:30:28,589 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] 09:30:28,589 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,589 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] 09:30:28,589 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:30:28,589 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] 09:30:28,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,601 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 09:30:28,608 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-49-0 of 1 operations, client version 12 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-49-0 ready 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 validated 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-49-0: success 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 candidate ready 09:30:28,609 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-49-0: canCommit - candidate rootPath: / 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-49-0: canCommit - messages: [] 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-49-0: preCommit - successfulFromPrevious: [] 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 prepared 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-49-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,610 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 submitted to persistence 09:30:28,612 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:30:28,612 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:30:28,613 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:30:28,613 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:28,614 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:30:28,615 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 09:30:28,620 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 09:30:28,620 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=12225820} is already disabled 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:30:28,621 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:30:28,623 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:30:28,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:30:28,624 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-49-0: commit - successfulFromPrevious: [] 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 changing state from COMMIT_PENDING to COMMITTED 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-49-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-647#832716521] 09:30:28,625 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2132797161] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 09:30:28,626 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:30:28,626 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:28,626 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:30:28,626 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2132797161] 09:30:28,626 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:30:28,626 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-2132797161] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,627 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 136.1 μs 09:30:28,628 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 09:30:28,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-1793815522] 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:28,629 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5a450218] 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:30:28,629 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5a450218] 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:30:28,629 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] for shard cars 09:30:28,629 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:28,629 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@680d57bc 09:30:28,629 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-49-0 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1365980893] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:30:28,630 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1365980893] 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,630 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:30:28,630 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 09:30:28,631 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#-1365980893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,631 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 152.8 μs 09:30:28,635 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,635 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-50-0 of 1 operations, client version 12 09:30:28,636 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,636 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-50-0 ready 09:30:28,637 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0 canCommit 09:30:28,637 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] resolved 09:30:28,637 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: successfully resolved all cohort actors 09:30:28,637 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0 finishCanCommit 09:30:28,637 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-50-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] 09:30:28,637 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 validated 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-50-0: success 09:30:28,638 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:28,638 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: canCommit returning result: true 09:30:28,638 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0 commit 09:30:28,638 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0 finish commit 09:30:28,638 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-50-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] 09:30:28,638 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 candidate ready 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-50-0: canCommit - candidate rootPath: / 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-50-0: canCommit - messages: [] 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-50-0: preCommit - successfulFromPrevious: [] 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 prepared 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,638 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-50-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 submitted to persistence 09:30:28,639 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:28,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-50-0: commit - successfulFromPrevious: [] 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 changing state from COMMIT_PENDING to COMMITTED 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-50-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%23-1619257954$a] 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:28,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 09:30:28,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:30:28,644 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-50-0: commit succeeded 09:30:28,644 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 09:30:28,644 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 84.02 μs 09:30:28,644 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#-1619257954)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:28,644 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@680d57bc 09:30:28,644 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:28,645 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:30:28,645 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 2 09:30:28,645 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 2 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 09:30:28,645 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-50-0 09:30:28,646 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:30:28,646 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 09:30:28,646 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 76.84 μs 09:30:28,646 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 09:30:28,646 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 09:30:28,647 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,647 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 09:30:28,647 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:30:28,647 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:30:28,647 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessagesSuccessful$] from Actor[akka://cluster-test/system/in-memory-journal#1040211245] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,647 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessageSuccess] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#-239609946] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,647 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:28,647 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:30:28,647 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:30:28,648 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 09:30:28,648 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:30:28,648 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,648 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,648 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1609306192] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1609306192] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,648 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 09:30:28,648 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=12] 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f6aaceb 09:30:28,648 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:28,648 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f6aaceb 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 09:30:28,648 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:28,648 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#88510008] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:28,864 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Failed to transfer leadership in 215.3 ms 09:30:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In unpauseLeader 09:30:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader transfer failed - sending PoisonPill 09:30:28,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leadership transfer expired 09:30:28,865 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:30:28,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-648: Closing 09:30:28,865 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:30:28,866 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:30:28,866 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:30:28,866 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:30:28,867 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 09:30:28,867 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:30:28,868 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:28,868 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:28,877 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:28,877 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:28,877 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:28,877 AM [cluster-test-akka.actor.default-dispatcher-12] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:28,880 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:28,894 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:28,894 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:29,890 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7816564040306612964] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:29,910 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:29,910 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:29,910 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3105653856253364115] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:29,912 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:29.914] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:29,929 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:29,929 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:29,933 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:29.934] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:29,963 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:29,969 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2462508085347699704] 09:30:29,969 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:29,971 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:29,971 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:29,972 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:29,972 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:29,984 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:29,984 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:29,984 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:29,988 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:29,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6058399443519731907] 09:30:29,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:29,998 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:29,998 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:29,999 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:29,999 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:30,29 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:30,68 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [515246498984194576] 09:30:30,69 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:30,71 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:30,71 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:30,74 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:30,78 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:30,78 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:30,80 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:30,84 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:30,85 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:30,87 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 09:30:30,94 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#770440631] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,94 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@68ff2746 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@32537200 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22c81420 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1714037430104, None)) 09:30:30,104 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037430103,0,0)) 09:30:30,105 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@32537200 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1930eb0c 09:30:30,105 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 09:30:30,105 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:30,107 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:30,109 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 09:30:30,117 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,117 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:30:30,118 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:30,119 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 09:30:30,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@59aedd73 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,120 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-775293547] 09:30:30,120 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:30,121 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 09:30:30,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-775293547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,121 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 09:30:30,123 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 09:30:30,123 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:30,124 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,124 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:30:30,124 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:30,124 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7f171da8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@359502c6 09:30:30,124 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1897899027 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,124 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-1897899027] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,125 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1358106990] 09:30:30,127 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7b25aeff 09:30:30,127 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1714037430124, None)) 09:30:30,128 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037430123,0,0)) 09:30:30,129 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@359502c6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2a8a59ab 09:30:30,129 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3fd1abd2 09:30:30,129 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 09:30:30,129 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:30,129 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,129 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:30,129 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#1358106990] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,129 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:30,129 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,129 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,130 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-2-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1200642578] 09:30:30,130 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:30,130 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 09:30:30,130 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 09:30:30,130 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 09:30:30,130 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 09:30:30,130 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#1200642578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-810777388] 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1306624672 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#1306624672] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,130 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-810777388] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:30,130 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 09:30:30,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:30,131 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:30,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 09:30:30,131 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 09:30:30,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:30,131 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:30:30,131 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,131 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 09:30:30,132 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 09:30:30,132 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 09:30:30,189 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#770440631] 09:30:30,189 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 09:30:30,189 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-775293547] 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:30,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:30:30,361 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:30:30,362 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 09:30:30,630 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:30,631 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,631 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 09:30:30,715 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:30,870 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:30,870 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,871 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 09:30:30,993 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:30,993 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,993 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:30,993 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,96 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:31,111 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:31,111 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,111 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 09:30:31,112 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,115 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 09:30:31,117 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:30:31,117 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:31,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:31,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:31,118 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 09:30:31,118 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:31,119 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 09:30:31,120 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:30:31,120 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:31,146 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,146 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] 09:30:31,146 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:31,146 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:31,147 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=12] 09:30:31,147 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:31,147 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=12] 09:30:31,147 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:31,147 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:31,147 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars 09:30:31,147 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 09:30:31,147 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 09:30:31,149 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,150 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483] 09:30:31,150 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,150 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 09:30:31,151 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}]]] 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, version=12] 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 09:30:31,152 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 3 batched modifications, ready: false 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483)] 09:30:31,152 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483)] 09:30:31,153 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66}] 09:30:31,153 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=6793826} is already disabled 09:30:31,155 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:31,156 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, size=66} 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:30:31,159 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:30:31,159 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=3] 09:30:31,159 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 09:30:31,159 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 09:30:31,159 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 09:30:31,160 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 1 batched modifications, ready: false 09:30:31,160 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483)] 09:30:31,160 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483)] 09:30:31,164 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with BatchedModificationsReply [numBatched=1] 09:30:31,164 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 DataExists operation succeeded 09:30:31,165 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 09:30:31,165 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 09:30:31,165 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 09:30:31,165 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 0 batched modifications, ready: true 09:30:31,165 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:0_5#-1420094483)] 09:30:31,166 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 09:30:31,167 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, client version 12 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: canCommit for member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: success 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 249 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 09:30:31,167 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 09:30:31,168 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}]]] 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=249} 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 2 09:30:31,173 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:31,174 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=249}] 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 1 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithSingleShard_member-1-shard-cars-testReadWriteTransactionWithSingleShard_shard-cars-member-2%3Adatastore-testReadWriteTransactionWithSingleShard%400%3A0_5%23-1420094483$f] 09:30:31,174 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1679446397] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 09:30:31,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1679446397] 09:30:31,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:30:31,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:31,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:31,175 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1679446397] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,175 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 09:30:31,175 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 152.6 μs 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 1 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 1 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=249} 09:30:31,176 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 09:30:31,176 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=12] 09:30:31,176 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:31,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1168277894] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 09:30:31,177 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1168277894] 09:30:31,177 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:31,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:30:31,177 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 09:30:31,177 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 88.40 μs 09:30:31,177 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 09:30:31,177 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1168277894] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,177 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:31,177 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:31,177 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] found - trying create transaction 09:30:31,177 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard)] 09:30:31,177 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 09:30:31,178 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 09:30:31,180 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_6#680214896, transactionId=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0, version=12] 09:30:31,180 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_6#680214896] 09:30:31,180 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:31,180 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_6#680214896)] 09:30:31,185 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 09:30:31,185 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:31,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,185 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 09:30:31,185 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 09:30:31,186 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,186 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 09:30:31,186 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:31,186 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:30:31,186 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:30:31,186 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:30:31,186 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:30:31,186 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 09:30:31,186 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:30:31,187 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,187 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1389100053] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1389100053] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,187 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,187 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 09:30:31,187 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 09:30:31,187 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 09:30:31,187 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-247396492] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@57ed98a0 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@57ed98a0 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:31,188 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:31,180 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_6#680214896] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,190 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-775293547] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,190 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#-775293547] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:31,470 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Failed to transfer leadership in 283.0 ms 09:30:31,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In unpauseLeader 09:30:31,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader transfer failed - sending PoisonPill 09:30:31,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leadership transfer expired 09:30:31,471 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:30:31,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-653: Closing 09:30:31,472 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:31,472 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:30:31,472 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:30:31,472 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:30:31,473 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 09:30:31,473 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:30:31,475 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:31,475 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:31,485 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:31,485 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:31,485 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:31,485 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:31,488 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:31,505 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:31,505 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:32,494 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2462508085347699704] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:32,523 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6058399443519731907] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:32,529 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:32,529 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:32,531 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:32.532] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:32,562 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:32,562 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:32,564 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:32.566] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:32,593 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:32,603 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-965513003604831686] 09:30:32,603 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:32,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:32,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:32,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:32,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:32,625 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 09:30:32,625 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testReadWriteTransactionWithSingleShard@0:1_6#680214896)] 09:30:32,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:32,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:32,630 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:32,638 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:32,652 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [812304425726464351] 09:30:32,652 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:32,654 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:32,654 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:32,655 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:32,655 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:32,675 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:32,697 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7187149664421501743] 09:30:32,705 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:32,705 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:32,705 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:32,705 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:32,705 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:32,715 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:32,717 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:32,719 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 09:30:32,751 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,752 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 09:30:32,758 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 09:30:32,758 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 09:30:32,758 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:32,758 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,758 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@37920796 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5612125e 09:30:32,759 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5a790b54 09:30:32,759 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,759 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:32,759 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,759 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,760 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1714037432759, None)) 09:30:32,760 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037432758,0,0)) 09:30:32,760 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5612125e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@102acede 09:30:32,761 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 09:30:32,761 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@36c1cc77 09:30:32,761 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:32,761 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,761 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029] 09:30:32,762 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 09:30:32,764 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 09:30:32,764 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:32,764 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 09:30:32,764 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 09:30:32,766 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#2107610782] 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#2107610782] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-2041163683 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-2041163683] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1023097702] 09:30:32,770 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-1023097702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,775 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-1835700617 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,775 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-1835700617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,781 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:32,783 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,785 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:32,785 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:32,786 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 09:30:32,786 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 09:30:32,786 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:32,786 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:32,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@252c0bb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f269c59 09:30:32,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2bd98a03 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1714037432787, None)) 09:30:32,787 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037432786,0,0)) 09:30:32,788 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:32,788 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6f269c59 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6005bb1 09:30:32,788 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 09:30:32,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:32,788 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:30:32,788 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 09:30:32,789 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:30:32,789 AM [cluster-test-shard-dispatcher-35] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 09:30:32,789 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 09:30:32,795 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:32,795 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:30:32,796 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 09:30:32,796 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:30:32,796 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:32,796 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 09:30:32,798 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6bf9d5d2 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:32,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:32,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 09:30:32,804 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 09:30:32,804 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 09:30:32,804 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 09:30:32,805 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#402814965 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2117672030] 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#-587769056] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-67145617 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#402814965] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-1522098420] 09:30:32,807 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#-67145617] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,811 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:32,811 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:32,811 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:32,812 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#2117672030] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,812 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#-1522098420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:32,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:32,813 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:32,813 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 09:30:32,813 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 09:30:32,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:32,813 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:30:32,814 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 09:30:32,814 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:32,814 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:30:32,814 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 09:30:32,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:32,816 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:32,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:32,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:30:32,816 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 09:30:32,816 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 09:30:32,816 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:32,817 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 09:30:32,817 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:30:32,817 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 09:30:32,838 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:32,838 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:32,839 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:33,29 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:30:33,29 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 09:30:33,30 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 09:30:33,38 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:33,38 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 09:30:33,38 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 09:30:33,38 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,38 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 09:30:33,39 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:33,39 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 09:30:33,39 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:30:33,39 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:30:33,39 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 09:30:33,39 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 09:30:33,298 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,298 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,298 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 09:30:33,318 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,318 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,319 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 09:30:33,412 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:33,431 AM [cluster-test-shard-dispatcher-31] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:33,548 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,548 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,548 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 09:30:33,579 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,579 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,579 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 09:30:33,620 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:33,620 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:33,620 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:33,620 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 09:30:33,620 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,621 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,698 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,710 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:33,788 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,788 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,788 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 09:30:33,789 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,791 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,792 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 09:30:33,794 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 09:30:33,794 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:30:33,795 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:33,795 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:33,795 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:33,797 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:33,798 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 09:30:33,798 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:33,869 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:30:33,869 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,869 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 09:30:33,869 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:33,876 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:33,876 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:33,876 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 09:30:33,876 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:33,876 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 09:30:33,877 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 09:30:33,877 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 09:30:33,877 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 09:30:33,878 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 09:30:33,878 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:30:33,879 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 09:30:33,879 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:33,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:33,894 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 09:30:33,894 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029] 09:30:33,902 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSnapshotOnRootOverwrite commit rateLimit adjusted to 79.76650015107607 09:30:33,903 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:33,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:33,903 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@67bce385] 09:30:33,903 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@67bce385] 09:30:33,903 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:33,903 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Queuing TransactionOperation 09:30:33,903 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Queuing TransactionOperation 09:30:33,903 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying 2 components for commit 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:33,904 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard cars 09:30:33,904 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Queuing TransactionOperation 09:30:33,904 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Readying transaction for shard default 09:30:33,904 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 Queuing TransactionOperation 09:30:33,904 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 canCommit 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@12201386] 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@12201386] 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] for shard default 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Shard default resolved to local data tree - adding local factory 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@a5feed8 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 = [ 1 ] 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,904 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, disabling transaction tracking 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:33,904 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 09:30:33,905 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 09:30:33,905 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,905 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] resolved 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 readyTransaction 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] resolved 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: successfully resolved all cohort actors 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finishCanCommit 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,905 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: success 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:33,906 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit returning result: true 09:30:33,906 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 commit 09:30:33,906 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 finish commit 09:30:33,906 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,906 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:33,906 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,906 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 09:30:33,906 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:33,907 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}]]] 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,907 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,908 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:30:33,908 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,908 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 09:30:33,909 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 282 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:33,909 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,909 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 09:30:33,909 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54}] 09:30:33,909 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,909 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 09:30:33,909 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 09:30:33,909 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=5279449} is already disabled 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=5587705} is already disabled 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 09:30:33,910 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:30:33,910 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 09:30:33,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}]]] 09:30:33,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:33,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:33,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 09:30:33,921 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:33,922 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 09:30:33,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 09:30:33,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:33,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 09:30:33,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 09:30:33,922 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 09:30:33,923 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,923 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:33,923 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:33,923 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-1575515029$a] 09:30:33,923 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-3847057] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 09:30:33,923 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-3847057] 09:30:33,923 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:33,923 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 93.52 μs 09:30:33,924 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-3847057] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 09:30:33,924 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7b3e6b02, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 09:30:33,925 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:33,926 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:33,926 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}]]] 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=282} 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 09:30:33,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, size=54} 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:33,928 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} does not match any client, pre-disabling client. 09:30:33,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:30:33,928 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=113} 09:30:33,929 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] 09:30:33,929 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:33,929 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-993076971] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 09:30:33,929 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$b] 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#951687396] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=113}] and replicatedToAllIndex is 0 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-993076971] 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 09:30:33,930 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit succeeded 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-993076971] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 09:30:33,930 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 42.33 μs 09:30:33,930 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#951687396] 09:30:33,931 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:33,931 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@40445772, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:33,931 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 09:30:33,931 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 09:30:33,931 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:33,931 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#951687396] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:33,931 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:33,931 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@15e53cad, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:33,931 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 09:30:33,931 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 09:30:33,931 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 207.4 μs 09:30:33,931 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 09:30:33,931 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:33,931 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:33,931 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:33,931 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 09:30:33,932 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 09:30:33,932 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] for shard default, trying to use DirectTransactionContextWrapper 09:30:33,932 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@a5feed8 09:30:33,932 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:33,932 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:33,932 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:33,932 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:30:33,932 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 09:30:33,932 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 09:30:33,932 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:33,932 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 09:30:33,932 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:33,933 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 1 09:30:33,933 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:33,933 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 1 09:30:33,933 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 09:30:33,933 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 1 data Simple{size=282} 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1979123387] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 0 for term 4 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=282}] and replicatedToAllIndex is 0 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=1, lastAppliedTerm=4, lastIndex=1, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 09:30:33,934 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:33,934 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1979123387] 09:30:33,935 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:33,935 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#1979123387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:33,935 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 89.47 μs 09:30:33,935 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:33,935 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=1, lastTerm=4, lastAppliedIndex=1, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@75855809, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:33,935 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 1 09:30:33,935 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 1 and term: 4 09:30:33,936 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 09:30:33,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 09:30:33,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:33,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:33,936 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 09:30:33,939 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:33,940 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:33,940 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:33,940 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:33,941 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] 09:30:33,941 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:33,941 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:33,941 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,146 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,146 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,146 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,146 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,146 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying 1 components for commit 09:30:34,146 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 Readying transaction for shard cars 09:30:34,146 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 = [ 1 ] 09:30:34,147 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,147 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 canCommit 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 validated 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: success 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 candidate ready 09:30:34,147 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - candidate rootPath: / 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: canCommit - messages: [] 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 prepared 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:34,148 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 submitted to persistence 09:30:34,150 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 09:30:34,152 AM [cluster-test-shard-dispatcher-13] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 216, lastApplied : 1, commitIndex : 1 09:30:34,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 09:30:34,153 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}]]] 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,154 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=222}] 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0: commit - successfulFromPrevious: [] 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$c] 09:30:34,155 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 successfully completed direct commit 09:30:34,155 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 committed successfully 09:30:34,155 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:34,155 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 102.0 μs 09:30:34,155 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 09:30:34,155 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-3-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,156 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,156 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,156 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying 1 components for commit 09:30:34,156 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 Readying transaction for shard cars 09:30:34,156 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 = [ 1 ] 09:30:34,156 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,156 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 canCommit 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 validated 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: success 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 candidate ready 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - candidate rootPath: / 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: canCommit - messages: [] 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: preCommit - successfulFromPrevious: [] 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 prepared 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,156 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 submitted to persistence 09:30:34,157 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 09:30:34,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 09:30:34,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:34,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 09:30:34,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 09:30:34,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=222} 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-2-0 09:30:34,158 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,158 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 61.94 μs 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}]]] 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,158 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 09:30:34,159 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,159 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,159 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=222}] 09:30:34,159 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 09:30:34,159 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 3, nextIndex: 4 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 3 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0: commit - successfulFromPrevious: [] 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$d] 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:30:34,160 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 09:30:34,160 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 committed successfully 09:30:34,160 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 successfully completed direct commit 09:30:34,160 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,160 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 09:30:34,160 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 71.60 μs 09:30:34,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,160 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-5-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,161 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,161 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,161 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,161 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying 1 components for commit 09:30:34,161 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 Readying transaction for shard cars 09:30:34,161 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 = [ 1 ] 09:30:34,161 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 canCommit 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 validated 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: success 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 candidate ready 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - candidate rootPath: / 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: canCommit - messages: [] 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: preCommit - successfulFromPrevious: [] 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 prepared 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 09:30:34,161 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 09:30:34,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 submitted to persistence 09:30:34,162 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 09:30:34,162 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 3 09:30:34,162 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:30:34,162 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 3 09:30:34,162 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 2 to 3 09:30:34,162 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 3 data Simple{size=222} 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-4-0 09:30:34,163 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 2 for term 4 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}]]] 09:30:34,163 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 55.61 μs 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 09:30:34,163 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,163 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,164 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=222}] 09:30:34,164 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=3 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 4, nextIndex: 5 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 4 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0: commit - successfulFromPrevious: [] 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$e] 09:30:34,164 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 successfully completed direct commit 09:30:34,164 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 committed successfully 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 09:30:34,164 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,164 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:30:34,164 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,164 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 56.82 μs 09:30:34,164 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 09:30:34,164 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,164 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-7-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,165 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,165 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,165 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,165 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying 1 components for commit 09:30:34,165 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 Readying transaction for shard cars 09:30:34,165 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 = [ 1 ] 09:30:34,165 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 canCommit 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 validated 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: success 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 candidate ready 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - candidate rootPath: / 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: canCommit - messages: [] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: preCommit - successfulFromPrevious: [] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 prepared 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 09:30:34,165 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 submitted to persistence 09:30:34,166 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:30:34,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 4 09:30:34,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 09:30:34,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 4 09:30:34,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 3 to 4 09:30:34,166 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 4 data Simple{size=222} 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-6-0 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 3 for term 4 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,167 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 5, nextIndex: 6 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 5 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0: commit - successfulFromPrevious: [] 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$f] 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:30:34,168 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 09:30:34,168 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 successfully completed direct commit 09:30:34,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 committed successfully 09:30:34,168 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,168 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 09:30:34,169 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 25.80 μs 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,169 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 77.38 μs 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-9-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,169 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:30:34,169 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,169 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=4 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,169 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,169 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying 1 components for commit 09:30:34,169 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 Readying transaction for shard cars 09:30:34,169 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 = [ 1 ] 09:30:34,169 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 canCommit 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 validated 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: success 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 candidate ready 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - candidate rootPath: / 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: canCommit - messages: [] 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: preCommit - successfulFromPrevious: [] 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 prepared 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 09:30:34,169 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 submitted to persistence 09:30:34,170 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 5 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 5 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 4 to 5 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 5 data Simple{size=222} 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-8-0 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 4 for term 4 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}]]] 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,171 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 76.40 μs 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 6, nextIndex: 7 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 6 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 09:30:34,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=222}] 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=5 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0: commit - successfulFromPrevious: [] 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,172 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$g] 09:30:34,173 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 09:30:34,173 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 09:30:34,173 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 09:30:34,173 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 successfully completed direct commit 09:30:34,173 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 committed successfully 09:30:34,173 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,173 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,173 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 54.24 μs 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-11-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,173 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,173 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,174 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying 1 components for commit 09:30:34,174 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 Readying transaction for shard cars 09:30:34,174 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 = [ 1 ] 09:30:34,174 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,174 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 canCommit 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 validated 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: success 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 candidate ready 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - candidate rootPath: / 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: canCommit - messages: [] 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: preCommit - successfulFromPrevious: [] 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 prepared 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 09:30:34,174 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 09:30:34,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 09:30:34,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 submitted to persistence 09:30:34,175 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 6 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 6 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 5 to 6 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 6 data Simple{size=222} 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-10-0 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 5 for term 4 09:30:34,176 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=6 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}]]] 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,177 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 7, nextIndex: 8 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 7 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0: commit - successfulFromPrevious: [] 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$h] 09:30:34,178 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 09:30:34,178 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 89.45 μs 09:30:34,178 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 09:30:34,179 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 09:30:34,179 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 committed successfully 09:30:34,179 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 successfully completed direct commit 09:30:34,179 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,179 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 09:30:34,179 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 45.19 μs 09:30:34,179 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,179 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,179 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-13-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,179 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,179 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,180 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,180 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,180 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying 1 components for commit 09:30:34,180 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 Readying transaction for shard cars 09:30:34,180 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 = [ 1 ] 09:30:34,180 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,180 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 canCommit 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 validated 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: success 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 candidate ready 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - candidate rootPath: / 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: canCommit - messages: [] 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: preCommit - successfulFromPrevious: [] 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 prepared 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 8, leaderLastIndex: 8, leaderSnapShotIndex: 6 09:30:34,180 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,181 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 09:30:34,181 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 submitted to persistence 09:30:34,181 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=222}] 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 7 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 7 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 6 to 7 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 7 data Simple{size=222} 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-12-0 09:30:34,183 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 6 for term 4 09:30:34,183 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 79.99 μs 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=7, prevLogTerm=4, leaderCommit=7, replicatedToAllIndex=6, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}]]] 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 7, entries to be added from: 0 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,183 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 09:30:34,184 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,184 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=8, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,184 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=Simple{size=222}] 09:30:34,184 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=7 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 8 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 8, nextIndex: 9 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 7 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 8 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0: commit - successfulFromPrevious: [] 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,185 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$i] 09:30:34,186 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 09:30:34,186 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 09:30:34,186 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 09:30:34,186 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 successfully completed direct commit 09:30:34,186 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 committed successfully 09:30:34,186 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,186 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,186 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 64.34 μs 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-15-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,186 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,186 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,186 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying 1 components for commit 09:30:34,186 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 Readying transaction for shard cars 09:30:34,186 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 = [ 1 ] 09:30:34,187 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,187 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 canCommit 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 validated 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: success 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 candidate ready 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - candidate rootPath: / 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: canCommit - messages: [] 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: preCommit - successfulFromPrevious: [] 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 prepared 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 9, leaderLastIndex: 9, leaderSnapShotIndex: 7 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 09:30:34,187 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 submitted to persistence 09:30:34,188 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 09:30:34,188 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 8 09:30:34,188 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 7, lastIndex(): 8 09:30:34,188 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 8 09:30:34,188 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 7 to 8 09:30:34,188 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 8 data Simple{size=222} 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-14-0 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 7 for term 4 09:30:34,189 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=8, prevLogTerm=4, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}]]] 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 09:30:34,189 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 71.37 μs 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,189 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=9, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,190 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=Simple{size=222}] 09:30:34,190 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=8 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 9 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 9, nextIndex: 10 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 9, lastAppliedIndex: 8 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 9 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0: commit - successfulFromPrevious: [] 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$j] 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 9, leaderSnapShotIndex: 8 09:30:34,191 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 09:30:34,192 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 successfully completed direct commit 09:30:34,192 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 committed successfully 09:30:34,192 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,192 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 09:30:34,192 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 52.20 μs 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-17-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,192 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,192 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,192 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying 1 components for commit 09:30:34,192 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 Readying transaction for shard cars 09:30:34,192 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 = [ 1 ] 09:30:34,192 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,192 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 canCommit 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 validated 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: success 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,192 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 candidate ready 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - candidate rootPath: / 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: canCommit - messages: [] 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: preCommit - successfulFromPrevious: [] 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 prepared 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 10, leaderLastIndex: 10, leaderSnapShotIndex: 8 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 09:30:34,193 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 submitted to persistence 09:30:34,194 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 9 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 9, context.getLastApplied(): 8, lastIndex(): 9 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 9 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 8 to 9 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 9 data Simple{size=222} 09:30:34,194 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-16-0 09:30:34,195 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 8 for term 4 09:30:34,195 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 73.88 μs 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=9, prevLogTerm=4, leaderCommit=9, replicatedToAllIndex=8, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}]]] 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 9, entries to be added from: 0 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,195 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,196 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=222}] 09:30:34,196 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=9 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 10 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 10, nextIndex: 11 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 9 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 10 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0: commit - successfulFromPrevious: [] 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$k] 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 09:30:34,196 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 09:30:34,197 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 09:30:34,197 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 successfully completed direct commit 09:30:34,197 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 committed successfully 09:30:34,197 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,197 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 09:30:34,197 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 59.07 μs 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-19-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,197 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,197 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,197 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying 1 components for commit 09:30:34,197 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 Readying transaction for shard cars 09:30:34,197 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 = [ 1 ] 09:30:34,197 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,197 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 canCommit 09:30:34,197 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,197 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,197 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 validated 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: success 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 candidate ready 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - candidate rootPath: / 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: canCommit - messages: [] 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: preCommit - successfulFromPrevious: [] 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 prepared 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 11, leaderLastIndex: 11, leaderSnapShotIndex: 9 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 09:30:34,198 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 submitted to persistence 09:30:34,199 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 10 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 9, lastIndex(): 10 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 10 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 9 to 10 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 10 data Simple{size=222} 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-18-0 09:30:34,200 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 9 for term 4 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}]]] 09:30:34,200 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 68.54 μs 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=222} 09:30:34,200 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 09:30:34,201 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,201 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=11, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,201 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=Simple{size=222}] 09:30:34,201 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=10 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 11, nextIndex: 12 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 11 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0: commit - successfulFromPrevious: [] 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$l] 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 09:30:34,202 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 09:30:34,202 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 successfully completed direct commit 09:30:34,202 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 committed successfully 09:30:34,202 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,202 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 09:30:34,203 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,203 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 57.00 μs 09:30:34,203 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,203 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-21-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,203 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:34,203 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:34,203 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:34,204 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 11 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 11 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 10 to 11 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 11 data Simple{size=222} 09:30:34,204 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-20-0 09:30:34,205 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,205 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 71.61 μs 09:30:34,205 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] 09:30:34,205 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:34,205 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,205 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 10 for term 4 09:30:34,205 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=11 09:30:34,206 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,409 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] for shard default, trying to use DirectTransactionContextWrapper 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@a5feed8 09:30:34,409 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,409 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying 2 components for commit 09:30:34,409 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard cars 09:30:34,409 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 09:30:34,409 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,410 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 Readying transaction for shard default 09:30:34,410 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 = [ 1 ] 09:30:34,410 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,410 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,410 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,410 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 canCommit 09:30:34,410 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handleReadyLocalTransaction for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,410 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying local modifications for Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] resolved 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransactionReadyReplyMapper - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 readyTransaction 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] resolved 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: successfully resolved all cohort actors 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finishCanCommit 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,410 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 resolved to actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: sending CanCommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Can committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 validated 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: canCommit for member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: success 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:34,411 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit returning result: true 09:30:34,411 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 commit 09:30:34,411 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 finish commit 09:30:34,411 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,411 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] 09:30:34,411 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: Sending CommitTransaction [transactionId=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 09:30:34,411 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 12, leaderLastIndex: 12, leaderSnapShotIndex: 10 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 12 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Committing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 candidate ready 09:30:34,412 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - candidate rootPath: / 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: canCommit - messages: [] 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: preCommit - successfulFromPrevious: [] 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 prepared 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Finishing commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 114 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 2 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 09:30:34,413 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 submitted to persistence 09:30:34,414 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 09:30:34,414 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 09:30:34,415 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=11, prevLogTerm=4, leaderCommit=11, replicatedToAllIndex=10, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}]]] 09:30:34,415 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,415 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 11, entries to be added from: 0 09:30:34,416 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 09:30:34,416 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}]]] 09:30:34,416 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 2 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:34,416 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=114} 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:30:34,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 12, nextIndex: 13 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 12, lastAppliedIndex: 11 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 12 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,418 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-cars-testSnapshotOnRootOverwrite%23-1504784358$m] 09:30:34,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,418 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 185.3 μs 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 11 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 29 09:30:34,419 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 2, nextIndex: 3 09:30:34,419 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:34,419 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:34,419 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 2 09:30:34,419 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 09:30:34,419 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit - successfulFromPrevious: [] 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 changing state from COMMIT_PENDING to COMMITTED 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSnapshotOnRootOverwrite_member-1-shard-default-testSnapshotOnRootOverwrite%23-1575515029$n] 09:30:34,420 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] 09:30:34,420 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] 09:30:34,420 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0: commit succeeded 09:30:34,420 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:34,420 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:34,420 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@43323428, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:34,420 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 128.5 μs 09:30:34,420 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 09:30:34,420 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:34,420 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 9 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 29 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 29 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@121e78d4, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 9 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 9 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:34,421 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 09:30:34,421 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:34,422 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:30:34,422 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@392171ed 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#-1575515029)] for shard default, trying to use DirectTransactionContextWrapper 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0 - Creating local component for shard default using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@a5feed8 09:30:34,422 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-23-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:34,423 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] 09:30:34,423 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:34,423 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:34,424 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1504784358] 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 12 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 12, context.getLastApplied(): 11, lastIndex(): 12 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 12 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from 11 to 12 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 12 data Simple{size=283} 09:30:34,424 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] 09:30:34,424 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 2 09:30:34,424 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:34,424 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 2 09:30:34,424 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 1 to 2 09:30:34,424 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 2 data Simple{size=114} 09:30:34,424 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,425 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-22-0 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:34,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,425 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=2 09:30:34,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 24.53 μs 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=114}] and replicatedToAllIndex is 1 09:30:34,425 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=2, lastAppliedTerm=4, lastIndex=2, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 7 09:30:34,425 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 113.1 μs 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:34,425 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,425 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=2, lastTerm=4, lastAppliedIndex=2, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@7ff038d5, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 2 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 2 and term: 4 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 7 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 7 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: fakeSnapshot purging log to 11 for term 4 09:30:34,426 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=12 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=12, term=4, payload=Simple{size=283}] and replicatedToAllIndex is 11 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=12, lastAppliedTerm=4, lastIndex=12, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 27 09:30:34,426 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=12, lastTerm=4, lastAppliedIndex=12, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@3de20d31, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 12 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 12 and term: 4 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 27 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 27 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 09:30:34,427 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 09:30:34,428 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 09:30:34,477 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#-139828261] 09:30:34,633 AM [cluster-test-shard-dispatcher-13] [WARN] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=12, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 207, lastApplied : 12, commitIndex : 12 09:30:34,639 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:30:34,680 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:34,680 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,680 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-2-shard-default-testSnapshotOnRootOverwrite 09:30:34,680 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 09:30:34,680 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onShutDown 09:30:34,680 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onShutDown 09:30:34,681 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 09:30:34,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 09:30:34,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:30:34,681 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:30:34,681 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 09:30:34,681 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:30:34,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 09:30:34,682 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 09:30:34,682 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:30:34,682 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:30:34,683 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:34,683 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 09:30:34,683 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@19468752 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5020d952 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5020d952 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@19468752 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 09:30:34,683 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 2 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 13, leaderLastIndex: 12, leaderSnapShotIndex: 12 09:30:34,683 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=12] 09:30:34,684 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:30:34,684 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=12] 09:30:34,684 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:34,684 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:34,684 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:30:34,685 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 09:30:34,685 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:30:34,685 AM [cluster-test-akka.actor.default-dispatcher-32] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#1629343699] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:34,949 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-default-testSnapshotOnRootOverwrite: Failed to transfer leadership in 265.8 ms 09:30:34,949 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In unpauseLeader 09:30:34,949 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader transfer failed - sending PoisonPill 09:30:34,949 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leadership transfer expired 09:30:34,950 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 09:30:34,950 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-661: Closing 09:30:34,951 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:30:34,989 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSnapshotOnRootOverwrite: Failed to transfer leadership in 305.3 ms 09:30:34,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In unpauseLeader 09:30:34,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader transfer failed - sending PoisonPill 09:30:34,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leadership transfer expired 09:30:34,990 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:30:34,990 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-660: Closing 09:30:34,990 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:30:34,990 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 09:30:34,990 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 09:30:34,990 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:30:34,991 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 09:30:34,991 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:30:34,991 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#84586183] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:34,992 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:34,992 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:30:35.000] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-827)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:30:35.000] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-827)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:30:35.000] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-827)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:30:35.000] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-827)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:35,21 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:35,21 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:36,9 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-965513003604831686] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:36,40 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [812304425726464351] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:36,42 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:36,42 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:36,43 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:36.045] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:36,52 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:36,52 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:36,54 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:36.056] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:36,66 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:36,71 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6524768479347633350] 09:30:36,71 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:36,72 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:36,72 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:36,73 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:36,73 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:36,83 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:36,83 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:36,84 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:36,87 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:36,95 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4672899127858482222] 09:30:36,95 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:36,99 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:36,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:36,105 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:36,105 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:36,117 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:36,130 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1203477590718971859] 09:30:36,130 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:36,134 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:36,134 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:36,134 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:36,134 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:36,143 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:36,145 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:36,146 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:30:36,160 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1088803964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,162 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:36,166 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:30:36,166 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:36,166 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3e304fe6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7885551f 09:30:36,166 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3d87d46b 09:30:36,166 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:30:36,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:36,167 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,167 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037436166, None)) 09:30:36,167 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037436165,0,0)) 09:30:36,167 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4f82bc3 09:30:36,167 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7885551f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@24d922be 09:30:36,167 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 09:30:36,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:36,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,168 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:36,169 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:36,171 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:30:36,177 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:36,177 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#180922156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,181 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:30:36,181 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4912abae to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ef70b06 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@206f4b2d 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037436182, None)) 09:30:36,182 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037436181,0,0)) 09:30:36,183 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ef70b06 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@58316b72 09:30:36,183 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 09:30:36,183 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@40e8db5a 09:30:36,183 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:36,183 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:36,184 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:30:36,184 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:30:36,184 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-37750328] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,184 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-578340921] 09:30:36,184 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1371903373 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:36,184 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1371903373] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,185 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:36,185 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:36,185 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:36,185 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:30:36,185 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:30:36,185 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 09:30:36,185 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:30:36,185 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 09:30:36,185 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 09:30:36,189 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-578340921] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,190 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280] 09:30:36,190 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:30:36,190 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:30:36,190 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1629094921] 09:30:36,190 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1274936236 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,190 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,190 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#1274936236] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,190 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-1629094921] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:36,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:36,191 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:36,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:36,191 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:30:36,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:30:36,191 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 09:30:36,191 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:30:36,191 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 09:30:36,191 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 09:30:36,212 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:36,215 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:36,216 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:36,216 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,216 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:36,217 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,218 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:36,222 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:36,233 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1088803964] 09:30:36,233 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:30:36,233 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280] 09:30:36,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:36,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:30:36,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 09:30:36,413 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:30:36,414 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:30:36,414 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 09:30:36,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:36,693 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,693 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 09:30:36,776 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:36,983 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:36,983 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:36,983 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 09:30:37,94 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:37,94 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,95 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,134 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,138 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:37,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,245 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 09:30:37,245 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,247 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:37,248 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:37,248 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:37,248 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 09:30:37,249 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 09:30:37,249 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:37,250 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:30:37,251 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:30:37,251 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:37,294 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testWriteTransactionWithSingleShard commit rateLimit adjusted to 79.76650015107607 09:30:37,294 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,294 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#180922156] 09:30:37,294 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,294 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,294 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,294 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,294 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:37,294 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying 1 components for commit 09:30:37,294 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Readying transaction for shard cars 09:30:37,295 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,295 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 canCommit 09:30:37,295 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=12] 09:30:37,295 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=12] 09:30:37,295 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars 09:30:37,295 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 09:30:37,295 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:37,295 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:37,296 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:37,296 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 09:30:37,296 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 directCommit called 09:30:37,296 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 sending 4 batched modifications, ready: true 09:30:37,296 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, disabling transaction tracking 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 09:30:37,300 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 4 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 of 4 operations, client version 12 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 ready 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 validated 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: success 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 candidate ready 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: canCommit - messages: [] 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 prepared 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,301 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 245 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}]]] 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,302 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 submitted to persistence 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 09:30:37,302 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:37,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 09:30:37,304 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 09:30:37,304 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62}] 09:30:37,304 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=5040685} is already disabled 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 09:30:37,305 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}]]] 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=245} 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:30:37,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=245}] 09:30:37,308 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 1, nextIndex: 2 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$a] 09:30:37,308 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#819272327] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 09:30:37,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:30:37,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:37,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#819272327] 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 successfully completed direct commit 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=12] 09:30:37,309 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 91.17 μs 09:30:37,309 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:37,309 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:37,309 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#819272327] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:37,309 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 09:30:37,309 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 09:30:37,310 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Queuing TransactionOperation 09:30:37,310 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 1 09:30:37,310 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:37,310 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 09:30:37,310 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:30:37,310 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data Simple{size=245} 09:30:37,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-192637306] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 09:30:37,311 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-192637306] 09:30:37,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:30:37,311 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=1 09:30:37,311 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:37,312 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#-192637306] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:37,312 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 159.3 μs 09:30:37,313 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:30:37,314 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_7#-861609468] 09:30:37,314 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_7#-861609468] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:37,315 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_7#-861609468, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0, version=12] 09:30:37,315 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:37,315 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_7#-861609468)] 09:30:37,318 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 ReadData operation succeeded 09:30:37,319 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:37,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1088803964] 09:30:37,319 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:30:37,319 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:37,319 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@28842b6e] 09:30:37,319 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@28842b6e] 09:30:37,319 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280)] for shard cars 09:30:37,319 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:37,319 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5e8f9812 09:30:37,319 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,319 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:37,319 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:37,319 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - creating WRITE_ONLY transaction context 09:30:37,319 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:37,319 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying 1 components for commit 09:30:37,320 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 Readying transaction for shard cars 09:30:37,320 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 directCommit called 09:30:37,320 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 sending 1 batched modifications, ready: true 09:30:37,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 09:30:37,320 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 canCommit 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Readying Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 of 1 operations, client version 12 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 ready 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 validated 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: canCommit for member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: success 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Committing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 candidate ready 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - candidate rootPath: / 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: canCommit - messages: [] 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 prepared 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Finishing commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 212 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 09:30:37,321 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 submitted to persistence 09:30:37,322 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}]]] 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=212} 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 2 09:30:37,323 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 09:30:37,324 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0: commit - successfulFromPrevious: [] 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithSingleShard_member-1-shard-cars-testWriteTransactionWithSingleShard$d] 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:37,325 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 09:30:37,325 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:37,325 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 101.2 μs 09:30:37,326 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=212}] 09:30:37,328 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 successfully completed direct commit 09:30:37,328 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 modifications completed with CommitTransactionReply [version=12] 09:30:37,328 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=212} 09:30:37,328 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:37,328 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:37,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:37,328 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 09:30:37,328 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 09:30:37,328 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Queuing TransactionOperation 09:30:37,329 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:30:37,329 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:30:37,329 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 09:30:37,329 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 89.76 μs 09:30:37,330 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 09:30:37,330 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_8#-1557167592] 09:30:37,330 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_8#-1557167592] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:37,331 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_8#-1557167592, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0, version=12] 09:30:37,331 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 09:30:37,331 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_8#-1557167592)] 09:30:37,333 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 DataExists operation succeeded 09:30:37,333 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 09:30:37,333 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:37,333 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:37,333 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] found - trying create transaction 09:30:37,333 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard)] 09:30:37,334 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Queuing TransactionOperation 09:30:37,334 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Creating transaction : member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 09:30:37,334 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_9#-2020594353] 09:30:37,335 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_9#-2020594353] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:37,335 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_9#-2020594353, transactionId=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0, version=12] 09:30:37,335 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:37,335 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_9#-2020594353)] 09:30:37,337 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 ReadData operation succeeded 09:30:37,337 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:30:37,337 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:37,337 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5e8f9812 09:30:37,337 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:30:37,338 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-1088803964] 09:30:37,338 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:30:37,338 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280] 09:30:37,338 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-1257704280] 09:30:37,390 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:37,391 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:37,391 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-666: Closing 09:30:37,391 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:37,391 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:37,391 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:37,392 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:37,400 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:37,400 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:37,401 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:37,401 AM [cluster-test-akka.actor.default-dispatcher-14] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:37,402 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:37,415 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:37,416 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:37,416 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:30:37,416 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:37,416 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:37,416 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:37,416 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:38,414 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6524768479347633350] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:38,434 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4672899127858482222] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:38,438 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:38,438 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:38,440 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:38.442] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:38,453 AM [reinstated-member2-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:38,466 AM [reinstated-member2-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [-3639713047790009453] 09:30:38,466 AM [reinstated-member2-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:38,468 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:38,468 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 09:30:38,468 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:38,469 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:38,480 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,481 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,483 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:30:38,485 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,485 AM [reinstated-member2-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:38,490 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:38,490 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:30:38,490 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@44013a8a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1f3c3781 09:30:38,491 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5a3eaa02 09:30:38,491 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037438490, None)) 09:30:38,491 AM [reinstated-member2-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:30:38,491 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,491 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,491 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037438489,0,0)) 09:30:38,491 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1f3c3781 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@56c6470c 09:30:38,491 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 09:30:38,492 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32752f02 09:30:38,492 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:38,492 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,492 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,493 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:30:38,493 AM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 09:30:38,493 AM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 09:30:38,493 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:30:38,494 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1580999326] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,494 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#1512434799] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,494 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1245284225] 09:30:38,496 AM [reinstated-member2-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1064001643 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,496 AM [reinstated-member2-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#410701814] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,497 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#410701814 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,497 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1591398106] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,500 AM [reinstated-member2-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 09:30:38,500 AM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:30:38,500 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:30:38,500 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 62 09:30:38,500 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 245 09:30:38,501 AM [reinstated-member2-shard-dispatcher-13] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:30:38,501 AM [reinstated-member2-shard-dispatcher-13] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, size=62} 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:30:38,501 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 1 09:30:38,501 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 09:30:38,501 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:30:38,501 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:38,501 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:38,501 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:38,501 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 09:30:38,502 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1245284225] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,502 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1064001643] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,502 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#1591398106] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5ba65f6f}, purged=MutableUnsignedLongSet{size=0}} 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 212 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 2 to 2 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 3.518 ms - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 09:30:38,504 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:30:38,504 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:30:38,504 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,504 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:30:38,505 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 09:30:38,505 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:30:38,505 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 09:30:38,505 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:30:38,542 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] 09:30:38,542 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:30:38,542 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1580999326] 09:30:38,542 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 09:30:38,542 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] 09:30:38,542 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 Queuing TransactionOperation 09:30:38,542 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:38,543 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1580999326, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@35e029b4] 09:30:38,543 AM [reinstated-member2-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1580999326, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@35e029b4] 09:30:38,543 AM [reinstated-member2-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0: Found primary ActorSelection[Anchor(akka://reinstated-member2/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-1580999326)] for shard cars 09:30:38,543 AM [reinstated-member2-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:38,543 AM [reinstated-member2-akka.actor.default-dispatcher-21] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@5c98493 09:30:38,543 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@3a53c7a5 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@3a53c7a5 09:30:38,544 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 09:30:38,544 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4691d4b0 09:30:38,544 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4691d4b0 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-671: Closing 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 09:30:38,544 AM [reinstated-member2-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-670: Closing 09:30:38,544 AM [reinstated-member2-shard-dispatcher-14] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:30:38,544 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:30:38,545 AM [reinstated-member2-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:30:38,545 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:38,545 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:38,545 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:30:38,545 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:38,545 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 09:30:38,545 AM [reinstated-member2-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,545 AM [reinstated-member2-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:38,545 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:30:38,545 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:30:38,546 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:30:38,546 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://reinstated-member2/user/$a#1142299980] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#1142299980] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,547 AM [reinstated-member2-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#-306946615] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:38,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:38,551 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:38,566 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:38,566 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:38,568 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:38.569] [reinstated-member2-akka.actor.internal-dispatcher-9] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:38,577 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:38,589 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4981273421783927785] 09:30:38,589 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:38,591 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:38,591 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:38,606 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:38,606 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:38,606 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:38,606 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:38,606 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:38,613 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:38,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [61710422256164243] 09:30:38,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:38,621 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:38,621 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:38,632 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:38,632 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:38,644 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:38,658 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4599499274029321302] 09:30:38,658 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:38,658 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:38,658 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:38,658 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:38,659 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:38,666 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,668 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,669 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 closeTransaction called 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:3_8#-1557167592)] 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 closeTransaction called 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:1_7#-861609468)] 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 closeTransaction called 09:30:38,686 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-cars-member-2:datastore-testWriteTransactionWithSingleShard@0:4_9#-2020594353)] 09:30:38,692 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-363463112] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,693 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,702 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:38,702 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 09:30:38,702 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:30:38,702 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@16c3d82c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a6ea291 09:30:38,703 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,703 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,703 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:38,703 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,707 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:38,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@dae07d5 09:30:38,707 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1714037438705, None)) 09:30:38,708 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037438704,0,0)) 09:30:38,708 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a6ea291 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@13c8b0ef 09:30:38,708 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 09:30:38,709 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,710 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:38,712 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c67fb58 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1242804741] 09:30:38,713 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 09:30:38,714 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1242804741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,715 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 09:30:38,719 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#288489912 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,719 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-309530462] 09:30:38,719 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#288489912] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,719 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#-309530462] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,722 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:38,722 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:38,723 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,723 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:30:38,723 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 09:30:38,723 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 09:30:38,724 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 09:30:38,724 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,724 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-367081665] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1d7aa4d2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11d453ac 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@15ebc54e 09:30:38,726 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1714037438726, None)) 09:30:38,727 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037438725,0,0)) 09:30:38,727 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11d453ac to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5e5856fa 09:30:38,727 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 09:30:38,727 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5e400853 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:38,728 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,729 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 09:30:38,729 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,729 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,729 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1121098117] 09:30:38,729 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 09:30:38,730 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 09:30:38,730 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1121098117] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,730 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#2134432251] 09:30:38,730 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1918208524 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,730 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#2134432251] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,730 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#-1918208524] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:38,730 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:38,730 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:38,730 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:38,730 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:30:38,731 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,731 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 09:30:38,731 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 09:30:38,731 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 09:30:38,731 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:38,731 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 09:30:38,737 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:38,777 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-363463112] 09:30:38,777 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 09:30:38,778 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-1242804741] 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:30:39,36 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 09:30:39,36 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 09:30:39,305 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:39,345 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:30:39,345 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,346 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 09:30:39,606 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:39,606 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,607 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,615 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:30:39,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,625 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 09:30:39,625 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=null 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,626 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 09:30:39,628 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=12] 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 09:30:39,628 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 09:30:39,629 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:39,629 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,629 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=12] 09:30:39,629 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:39,629 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:39,629 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 09:30:39,629 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 09:30:39,629 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 09:30:39,629 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:39,632 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 09:30:39,632 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-367081665] 09:30:39,633 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Queuing TransactionOperation 09:30:39,633 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:39,633 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=12] 09:30:39,633 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=12] 09:30:39,633 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars 09:30:39,633 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 09:30:39,633 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 09:30:39,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, disabling transaction tracking 09:30:39,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 09:30:39,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 09:30:39,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:39,634 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:39,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 09:30:39,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Creating transaction : member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 09:30:39,635 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_10#862477529] 09:30:39,635 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_10#862477529] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:39,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}]]] 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Number of entries to be appended = 1 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74} 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 09:30:39,636 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_10#862477529, transactionId=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0, version=12] 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Log size is now 1 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:39,636 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:39,636 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_10#862477529)] 09:30:39,636 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, size=74}] 09:30:39,637 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done true 09:30:39,641 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 ReadData operation succeeded 09:30:39,641 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:39,642 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,642 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-676: Closing 09:30:39,642 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 09:30:39,642 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,642 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:39,642 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:39,651 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:39,654 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:39,654 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:39,654 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:39,657 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:30:39,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:39,682 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:30:39,782 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,783 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithCreateTxFailureDueToNoLeader commit rateLimit adjusted to 79.76650015107607 09:30:39,783 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,783 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:39,783 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:39,783 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 09:30:39,783 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,783 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 09:30:39,783 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:30:39,784 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying 1 components for commit 09:30:39,784 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Readying transaction for shard cars 09:30:39,784 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Queuing TransactionOperation 09:30:39,784 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 canCommit 09:30:39,804 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:30:39,834 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:30:39,836 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 09:30:39,836 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 09:30:39,864 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:30:39,894 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:30:39,894 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] found - trying create transaction 09:30:39,894 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)] 09:30:39,914 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:30:39,914 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:30:39,914 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:30:39,914 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 4 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:39,915 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:39,915 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 09:30:39,924 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader)]] after [10 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:30:39,954 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:30:39,954 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-367081665] 09:30:39,954 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:39,955 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 09:30:40,134 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:30:40,134 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:40,134 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 09:30:40,135 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=0, lastLogTerm=3] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:40,174 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 09:30:40,174 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 09:30:40,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0: Find primary for shard cars failed org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:40,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 Creating NoOpTransaction because of error org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:40,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:40,175 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:40,176 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:30:40,176 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:40,177 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader - waiting on 1 shards 09:30:40,177 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onShutDown 09:30:40,177 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:30:40,177 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 09:30:40,178 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All shards shutdown - sending PoisonPill to self 09:30:40,178 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:30:40,179 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:40,179 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:30:40,179 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:30:40,179 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:40,179 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: shutting down 09:30:40,180 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:30:40,179 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-367081665] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#-367081665] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:40,180 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:40,180 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:40,655 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4981273421783927785] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:41,195 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [61710422256164243] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:41,196 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:41,197 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:41,200 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:41.201] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:41,207 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:41,207 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:41,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:41.211] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:41,229 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:41,241 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8118999755399702582] 09:30:41,242 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:41,243 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:41,243 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:41,244 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:41,244 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:41,261 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:41,261 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:41,261 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:41,262 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:41,269 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1585721362833739910] 09:30:41,270 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:41,271 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:41,271 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:41,272 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:41,272 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:41,291 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:41,305 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3438906896956329370] 09:30:41,306 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:41,311 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:41,311 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:41,317 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:41,317 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:41,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:41,322 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:41,324 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:41,326 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:41,328 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,332 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#742228651] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,332 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:41,334 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 09:30:41,334 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:41,334 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3964d7e9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3b746f78 09:30:41,334 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3b575e21 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:41,335 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1714037441334, None)) 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,335 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,336 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037441333,0,0)) 09:30:41,336 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@599e19c3 09:30:41,336 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3b746f78 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5294fe5b 09:30:41,336 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 09:30:41,337 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:41,338 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:41,340 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,341 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:41,341 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,345 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,345 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,345 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,345 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-871943007] 09:30:41,345 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,345 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 09:30:41,345 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:41,346 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:30:41,346 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 09:30:41,346 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-871943007] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1257065365] 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#1257065365] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#41343787 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-2117110802] 09:30:41,347 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-125286447] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,349 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,349 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,349 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649] 09:30:41,349 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-125286447 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,349 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:41,350 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:41,350 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 09:30:41,350 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 09:30:41,350 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,350 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,350 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#41343787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,350 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:30:41,350 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-2117110802] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,350 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,350 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 09:30:41,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 09:30:41,352 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:30:41,352 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,352 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 09:30:41,353 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 09:30:41,354 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:41,354 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1a69597 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@339731e8 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,354 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,354 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@c622999 09:30:41,354 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1714037441354, None)) 09:30:41,360 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037441353,0,0)) 09:30:41,360 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@339731e8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6ff1d5bc 09:30:41,360 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1fb57e4d 09:30:41,360 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1587386583] 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-519463561] 09:30:41,361 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 09:30:41,362 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 09:30:41,362 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:30:41,362 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:30:41,362 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#6313986] 09:30:41,362 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-106433905 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,362 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#102491578] 09:30:41,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1587386583] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-519463561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-106433905] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#6313986] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#102491578] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:41,363 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-886954005 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,363 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-886954005] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 09:30:41,363 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 09:30:41,368 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:30:41,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,369 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,369 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,369 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:30:41,369 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:30:41,369 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 09:30:41,369 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 09:30:41,373 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:41,374 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:41,411 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#742228651] 09:30:41,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 09:30:41,411 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-871943007] 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:30:41,635 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,636 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,636 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 09:30:41,636 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 09:30:41,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:41,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:30:41,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 09:30:41,665 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,665 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 09:30:41,666 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:41,666 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:30:41,666 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:30:41,666 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:30:41,666 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 09:30:41,666 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 09:30:41,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:41,895 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:41,895 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 09:30:41,905 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:41,905 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:41,905 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 09:30:41,929 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:41,939 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:42,195 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:42,195 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,195 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 09:30:42,195 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:42,195 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,195 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 09:30:42,257 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:42,257 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:42,258 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:42,258 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,258 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,259 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,337 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:42,455 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:42,455 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,455 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 09:30:42,455 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,458 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 09:30:42,460 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:42,460 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 09:30:42,461 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 09:30:42,461 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,461 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 09:30:42,462 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:30:42,462 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:42,462 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:42,462 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:30:42,462 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:42,515 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:30:42,515 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,515 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 09:30:42,515 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,517 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 09:30:42,519 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 09:30:42,519 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 09:30:42,520 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 09:30:42,520 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,520 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 09:30:42,520 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:30:42,520 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:42,521 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:42,521 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 09:30:42,521 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:30:42,521 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:42,571 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#742228651] 09:30:42,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 09:30:42,571 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649] 09:30:42,572 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 762204161, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@4324e2ed} 09:30:42,572 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithMultipleShards commit rateLimit adjusted to 79.76650015107607 09:30:42,573 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:42,573 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] 09:30:42,573 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:30:42,573 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:42,573 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:30:42,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:30:42,573 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] 09:30:42,573 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:42,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] for shard people 09:30:42,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:30:42,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:30:42,573 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:42,573 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:30:42,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] for shard cars 09:30:42,574 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying 2 components for commit 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:30:42,574 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:42,574 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:30:42,574 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 Readying transaction for shard people 09:30:42,574 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 = [ 1 ] 09:30:42,574 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction called 09:30:42,574 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 09:30:42,574 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards)] 09:30:42,574 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards)] 09:30:42,574 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 09:30:42,574 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 canCommit 09:30:42,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 09:30:42,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,577 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@71981d37 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Readying Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 of 1 operations, client version 12 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:42,578 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 ready 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, disabling transaction tracking 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,579 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:30:42,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} 09:30:42,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6752735d 09:30:42,580 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:30:42,580 AM [cluster-test-shard-dispatcher-29] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:42,581 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,581 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}]]] 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 09:30:42,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:42,584 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 readyTransaction 09:30:42,584 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649)] resolved 09:30:42,584 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:42,584 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,584 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done true 09:30:42,584 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,584 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 09:30:42,585 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74}] 09:30:42,585 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:30:42,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:42,587 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 09:30:42,587 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=8750730} is already disabled 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=10387777} is already disabled 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 09:30:42,588 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:42,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}]]] 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,594 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:42,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:42,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 2 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 0 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 0 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, size=74} 09:30:42,595 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a cohort Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:30:42,595 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,595 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:42,595 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@245a0028 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1124) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$4.onFailure(ThreePhaseCommitCohortProxy.java:140) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1124) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:102) at org.opendaylight.controller.cluster.datastore.ThreePhaseCommitCohortProxy$3.onComplete(ThreePhaseCommitCohortProxy.java:95) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:30:42,596 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 abort 09:30:42,596 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0: a abort cohort path Future failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:42,596 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 09:30:42,596 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:30:42,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76}] 09:30:42,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 1 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} has no callback 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:42,598 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:42,598 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:42,600 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Commit index set to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Setting last applied to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Moving last applied index from 0 to 1 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0}, size=76} 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:30:42,600 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:30:42,606 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@245a0028 is no longer running 09:30:42,606 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,607 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 09:30:42,607 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 09:30:42,607 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 09:30:42,607 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,607 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,607 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:30:42,607 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:30:42,607 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:30:42,607 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:30:42,613 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:30:42,613 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:42,614 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:30:42,614 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 09:30:42,614 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@479d4eb2 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e23a10f 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6e23a10f 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 09:30:42,614 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:42,615 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:42,616 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 09:30:42,616 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:30:42,617 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:42,617 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-682924694] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:42,617 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:42,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 09:30:42,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:42,875 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Failed to transfer leadership in 260.7 ms 09:30:42,875 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In unpauseLeader 09:30:42,875 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader transfer failed - sending PoisonPill 09:30:42,875 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leadership transfer expired 09:30:42,876 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:30:42,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-684: Closing 09:30:42,876 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:30:43,26 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:30:43,26 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#1827654649]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:30:43,26 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:30:43,26 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Failed to transfer leadership in 412.1 ms 09:30:43,26 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In unpauseLeader 09:30:43,26 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader transfer failed - sending PoisonPill 09:30:43,26 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-683: Closing 09:30:43,27 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 1 pending queued transactions 09:30:43,27 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:30:43,27 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:30:43,27 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 09:30:43,27 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:30:43,28 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:43,28 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:43,33 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:43,33 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:43,33 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:43,34 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:43,46 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:43,46 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:44,47 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8118999755399702582] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:44,61 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:44,61 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:44,63 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:44.064] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:44,71 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1585721362833739910] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:44,80 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:44,80 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:44,83 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:44.086] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:44,105 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:30:44,111 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1437899744657142806] 09:30:44,111 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:44,114 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:44,114 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:44,114 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:44,114 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:44,129 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:44,129 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:44,129 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:44,136 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:44,147 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2483016560843813905] 09:30:44,147 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:44,149 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:44,149 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:44,163 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:44,163 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:44,188 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:44,198 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:44,200 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:44,224 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:44,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2555209328624220199] 09:30:44,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:44,225 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:44,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:44,225 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:44,226 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:44,227 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:30:44,234 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:44,261 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:44,264 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,284 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:30:44,284 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,284 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@62362ae0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2292f23e 09:30:44,284 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@29452c23 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,284 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,285 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037444284, None)) 09:30:44,285 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037444283,0,0)) 09:30:44,285 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6f3d14db 09:30:44,285 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2292f23e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@78b8abad 09:30:44,285 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:30:44,286 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:44,286 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:44,289 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:44,289 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,289 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:44,290 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:44,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,291 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:30:44,291 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:30:44,292 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:30:44,292 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:44,292 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:30:44,293 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1953378832 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1953378832] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-957468136] 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-957468136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-166287684] 09:30:44,294 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1921983889 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,301 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] 09:30:44,306 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#1921983889] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,306 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-166287684] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,306 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:44,306 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:44,306 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:30:44,306 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 09:30:44,306 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:44,306 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:44,307 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:30:44,307 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 09:30:44,316 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#32011897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,319 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:44,320 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:44,320 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:30:44,320 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,320 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,320 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:44,320 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:44,321 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:30:44,333 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:30:44,333 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:44,333 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@65d5a9df to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6dbe3856 09:30:44,334 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@252ca1b1 09:30:44,334 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037444333, None)) 09:30:44,335 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037444332,0,0)) 09:30:44,335 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6dbe3856 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@62a9d7ed 09:30:44,335 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:30:44,337 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:44,338 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:30:44,338 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:44,338 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3b45613e 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-1310298889] 09:30:44,339 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 09:30:44,340 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:30:44,340 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:30:44,341 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:30:44,341 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:30:44,341 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#-1310298889] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,341 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#933976736] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,341 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1196797861 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:44,341 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1196797861] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,342 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#323020893] 09:30:44,342 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#323020893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,342 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-298515054 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 09:30:44,342 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1995913302] 09:30:44,345 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,345 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:44,345 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#933976736] 09:30:44,346 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#-298515054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,346 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#1995913302] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:30:44,350 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 09:30:44,350 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:44,351 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,351 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:30:44,351 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 09:30:44,351 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:44,351 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:30:44,386 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] 09:30:44,386 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:30:44,386 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,552 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,552 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:30:44,553 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:30:44,572 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:30:44,572 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 09:30:44,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:30:44,831 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:44,831 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 09:30:44,872 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:30:44,872 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:30:44,872 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 09:30:45,132 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:30:45,132 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,132 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 09:30:45,132 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:45,132 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,133 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,134 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:45,144 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,144 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:30:45,151 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,151 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 09:30:45,152 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,152 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,153 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:45,153 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,154 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 09:30:45,155 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:45,155 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 09:30:45,156 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,156 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 09:30:45,156 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,156 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:45,156 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:45,156 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,157 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 09:30:45,157 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:30:45,158 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,233 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:45,274 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,275 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 09:30:45,361 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:30:45,361 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,361 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 09:30:45,362 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,362 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,364 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:45,364 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,372 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:45,372 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 09:30:45,373 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 09:30:45,373 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,377 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,377 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,377 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,377 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,378 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,394 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,446 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] 09:30:45,446 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 09:30:45,447 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] 09:30:45,447 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:45,450 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:45,452 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d1d478d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a62392b 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1213345839] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5e5da6bf 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037445462, None)) 09:30:45,462 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037445461,0,0)) 09:30:45,463 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5a62392b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@57e17895 09:30:45,463 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:30:45,463 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testLeadershipTransferOnShutdown commit rateLimit adjusted to 79.76650015107607 09:30:45,463 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#32011897] 09:30:45,463 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,463 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,463 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#32011897] 09:30:45,463 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,464 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying 2 components for commit 09:30:45,464 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard cars 09:30:45,464 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,464 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Readying transaction for shard people 09:30:45,464 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:45,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:30:45,464 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,464 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 2 ] 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 2 batched modifications, ready: true 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 09:30:45,464 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] for shard people 09:30:45,464 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 = [ 1 ] 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction called 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown)] 09:30:45,465 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:30:45,465 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,465 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,466 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c7b48c0 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,468 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:30:45,468 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1379540942] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,468 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1497125207 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,469 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-1497125207] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,469 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#813901857] 09:30:45,469 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,469 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,469 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:30:45,469 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 2 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 2 operations, client version 12 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, disabling transaction tracking 09:30:45,470 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#-1341193459 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,470 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#995947536] 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,470 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 of 1 operations, client version 12 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,471 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 ready 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:45,471 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,471 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#813901857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,473 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,473 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:45,474 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:45,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,474 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:45,474 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 3 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=3, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 09:30:45,475 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 09:30:45,475 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 09:30:45,475 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,476 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:30:45,476 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 09:30:45,476 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:30:45,476 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,476 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,476 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,476 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,476 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:30:45,478 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,478 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 09:30:45,478 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,478 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,478 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:45,479 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,479 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,479 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:30:45,479 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=8964344} is already disabled 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=10672223} is already disabled 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,480 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,480 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:30:45,480 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,482 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 09:30:45,482 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:45,482 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 readyTransaction 09:30:45,482 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,486 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,487 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:30:45,487 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 09:30:45,487 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:30:45,487 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:30:45,487 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=3, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,487 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:45,487 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:30:45,487 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,487 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,487 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:30:45,487 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:30:45,488 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,488 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,488 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,488 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:30:45,488 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,489 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:30:45,489 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,489 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,489 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}]]] 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59} 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,490 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} does not match any client, pre-disabling client. 09:30:45,491 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:30:45,491 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:45,494 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, size=59}] 09:30:45,495 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:30:45,495 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:30:45,517 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,517 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,517 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:45,517 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:45,517 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] found - creating WRITE_ONLY transaction context 09:30:45,517 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:30:45,517 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying 1 components for commit 09:30:45,517 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 Readying transaction for shard cars 09:30:45,517 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 directCommit called 09:30:45,517 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 sending 1 batched modifications, ready: true 09:30:45,517 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown)] 09:30:45,518 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] 09:30:45,518 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:30:45,519 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,519 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,520 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying 1 batched modifications for Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,520 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Readying Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 of 1 operations, client version 12 09:30:45,520 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,520 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 ready 09:30:45,520 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 is scheduled for canCommit step 09:30:45,520 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 09:30:45,521 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1865706763] 09:30:45,521 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,521 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 09:30:45,521 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:30:45,522 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:30:45,522 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] 09:30:45,522 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 canCommit 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861)] resolved 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903)] resolved 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: successfully resolved all cohort actors 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finishCanCommit 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861)] 09:30:45,522 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861)] 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1561254d 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,522 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,524 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,524 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,524 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 09:30:45,524 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:45,524 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 09:30:45,524 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,524 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,525 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,526 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:45,526 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903)] 09:30:45,526 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903)] 09:30:45,534 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Can committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,534 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,534 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 validated 09:30:45,534 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:45,534 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: success 09:30:45,534 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:45,534 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit returning result: true 09:30:45,534 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 commit 09:30:45,534 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 finish commit 09:30:45,534 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861)] 09:30:45,534 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861)] 09:30:45,535 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903)] 09:30:45,535 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903)] 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 169 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 validated 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: canCommit for member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: success 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,538 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 candidate ready 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - candidate rootPath: / 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: canCommit - messages: [] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 prepared 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 scheduled for commit step 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: false 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 228 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,539 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Committing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,539 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,539 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 candidate ready 09:30:45,540 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:30:45,540 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:45,540 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,540 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 submitted to persistence 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: canCommit - messages: [] 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 prepared 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Finishing commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:45,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 09:30:45,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:45,541 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 09:30:45,541 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 submitted to persistence 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 09:30:45,543 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 09:30:45,543 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,544 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=169} 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=228} 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}]]] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=173} 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 2 09:30:45,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] 09:30:45,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] 09:30:45,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=173}] 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-people-testLeadershipTransferOnShutdown%23895530903$g] 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-918777442] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown%23-395255861$f] 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 3 09:30:45,546 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-583819887] with name member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:45,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-583819887] 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-918777442] 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0: commit - successfulFromPrevious: [] 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$c] 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1561254d 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 139.5 μs 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 33.42 μs 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 98.00 μs 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-583819887] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,547 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-918777442] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-3-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 09:30:45,548 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 09:30:45,549 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 09:30:45,549 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:45,549 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:30:45,549 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 4 09:30:45,549 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 09:30:45,550 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: commit succeeded 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 09:30:45,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=12] 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}]]] 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 2 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=169}] is present: existingEntryTerm: 4 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=228}] is present: existingEntryTerm: 4 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 2 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=173} 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#652814898] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 3 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1754719098] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data Simple{size=169} 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,551 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1844125765] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-775159424] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 09:30:45,551 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1844125765] 09:30:45,551 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 canCommit 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 60.54 μs 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#1844125765] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1754719098] 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 successfully completed direct commit 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#652814898] 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 3 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 45.27 μs 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=228} 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1754719098] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 69.25 μs 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-775159424] 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 28.25 μs 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,552 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 54.10 μs 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 0 for term 4 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,552 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,553 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 09:30:45,553 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 32.17 μs 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,553 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,553 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:30:45,552 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:30:45,554 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,554 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:30:45,555 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 4 - switching to Follower 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 2, lastTerm: 4 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-692: Closing 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 09:30:45,556 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:30:45,556 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,556 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 09:30:45,556 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,557 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,563 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,563 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-3-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,563 AM [cluster-test-shard-dispatcher-32] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-3-shard-cars-testLeadershipTransferOnShutdown in 41.28 ms 09:30:45,563 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 09:30:45,563 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:30:45,564 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,564 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:45,564 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-395255861] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1953378832] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#-1953378832] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:45,564 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:30:45,564 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 09:30:45,564 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:30:45,565 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:30:45,565 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#32011897] 09:30:45,565 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 09:30:45,565 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,565 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Queuing TransactionOperation 09:30:45,565 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:45,565 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:30:45,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] for shard cars 09:30:45,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] found - trying create transaction 09:30:45,566 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown)] 09:30:45,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:30:45,567 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Creating transaction : member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 09:30:45,567 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_11#1386365124] 09:30:45,568 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_11#1386365124, transactionId=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0, version=12] 09:30:45,568 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:45,568 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_11#1386365124)] 09:30:45,571 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 ReadData operation succeeded 09:30:45,571 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:30:45,571 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1213345839] 09:30:45,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:45,571 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 Queuing TransactionOperation 09:30:45,571 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1379540942, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6949defb] 09:30:45,571 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1379540942, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6949defb] 09:30:45,571 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#1379540942)] for shard cars 09:30:45,571 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:30:45,571 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractTransactionContextFactory - Tx member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4011f097 09:30:45,571 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b0d6a8 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@45b0d6a8 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,572 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:45,572 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,576 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: fakeSnapshot purging log to 1 for term 5 09:30:45,576 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-3-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,576 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,576 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,576 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: 2, lastIndex: 2, isVoting: true 09:30:45,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 09:30:45,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,577 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 6 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,578 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,579 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:30:45,579 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4]" message is greater than leader's term 5 - switching to Follower 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=null 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 2, lastTerm: 4 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=6, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 6 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-3-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-696: Closing 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: removing frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 09:30:45,583 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 09:30:45,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 09:30:45,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:30:45,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,585 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,585 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 6 09:30:45,585 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:30:45,585 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,585 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [member-2-frontend-datastore-testLeadershipTransferOnShutdown] 09:30:45,585 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,585 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,585 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:30:45,585 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 09:30:45,585 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,586 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] RaftActorLeadershipTransferCohort - member-3-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 14.93 ms 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:30:45,587 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:30:45,587 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:30:45,587 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,587 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:45,587 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:30:45,587 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:45,588 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:45,588 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:45,588 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:30:45,588 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:45,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:45,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,588 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,588 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35b05d9a 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@35b05d9a 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:30:45,588 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:45,588 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 09:30:45,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:45,589 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:30:45,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:45,589 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:45,754 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#895530903] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:46,152 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:30:46,659 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testLeadershipTransferOnShutdown: Failed to transfer leadership in 1.070 s 09:30:46,659 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: In unpauseLeader 09:30:46,659 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: leader transfer failed - sending PoisonPill 09:30:46,659 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer expired 09:30:46,660 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:30:46,660 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-697: Closing 09:30:46,660 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:46,661 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:30:46,661 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:46,661 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:46,661 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:30:46,661 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:30:46,661 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:46,662 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a0d0 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@29a0d0 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:46,662 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:56,672 AM [cluster-test-shard-dispatcher-34] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Failed to transfer leadership in 10.01 s 09:30:56,672 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In unpauseLeader 09:30:56,672 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer failed - sending PoisonPill 09:30:56,672 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer expired 09:30:56,673 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 09:30:56,673 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-691: Closing 09:30:56,673 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:30:56,673 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:30:56,673 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:30:56,674 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:56,674 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:30:56,674 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:30:56,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:56,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:56,680 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:56,680 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:56,680 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:56,680 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:56,694 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:56,694 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:57,693 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1437899744657142806] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:57,712 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:57,712 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:30:57,713 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2483016560843813905] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:30:57,714 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:57.715] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:57,721 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:30:57,721 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:30:57,723 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:30:57.724] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:57,739 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:57,745 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2249789142914040770] 09:30:57,745 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:30:57,746 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:30:57,746 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:30:57,747 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:57,747 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:57,759 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:30:57,759 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:30:57,759 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:30:57,762 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:30:57,767 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7869945781402327951] 09:30:57,767 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:30:57,769 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:57,769 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:30:57,780 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:57,780 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:57,783 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 closeTransaction called 09:30:57,783 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-cars-member-2:datastore-testTransactionWithCreateTxFailureDueToNoLeader@0:0_10#862477529)] 09:30:57,783 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 closeTransaction called 09:30:57,783 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2557/), Path(/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-cars-member-2:datastore-testLeadershipTransferOnShutdown@0:2_11#1386365124)] 09:30:57,808 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:30:57,822 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:30:57,824 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:57,826 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4841272986335561335] 09:30:57,826 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:30:57,827 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:30:57,827 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:30:57,829 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:30:57,829 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:30:57,837 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:57,838 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:57,840 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,856 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,859 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:30:57,860 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:30:57,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,867 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 09:30:57,867 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:57,867 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@25428689 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e4fc9f2 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,867 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6c57a2c3 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,867 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,867 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1714037457867, None)) 09:30:57,868 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037457866,0,0)) 09:30:57,868 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@9ba8f34 09:30:57,868 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1e4fc9f2 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7afb89b6 09:30:57,868 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 09:30:57,868 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:57,869 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:30:57,871 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191] 09:30:57,873 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:30:57,873 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1040891279 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,873 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-475148884] 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#143734310 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1367296663] 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1040891279] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-475148884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#143734310] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1367296663] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:57,874 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:57,874 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:30:57,874 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,874 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:30:57,876 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:30:57,876 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:57,876 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:57,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 09:30:57,876 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 09:30:57,877 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:57,877 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:57,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:30:57,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,877 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:30:57,881 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,882 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:30:57,885 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 09:30:57,886 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,886 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,886 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:30:57,886 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,886 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@8139c20 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2a841997 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@63415531 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1714037457886, None)) 09:30:57,886 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037457885,0,0)) 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7d6c2717 09:30:57,887 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2a841997 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4420de61 09:30:57,887 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,887 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] 09:30:57,888 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,888 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,888 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,888 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1012721072] 09:30:57,888 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,888 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:30:57,888 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1012721072] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,888 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:30:57,888 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:30:57,889 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1007736338] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,890 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1803505734 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,890 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1007736338 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,891 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1377649897] 09:30:57,891 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:30:57,891 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#1803505734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,891 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1377649897] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,891 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1157294610] 09:30:57,892 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1157294610] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:57,893 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:30:57,893 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,893 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 09:30:57,893 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:30:57,893 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:30:57,893 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:30:57,937 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] 09:30:57,937 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:30:57,937 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:58,127 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:30:58,127 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:30:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:30:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 09:30:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:30:58,186 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,186 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 09:30:58,187 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:30:58,187 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:30:58,187 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:30:58,187 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:30:58,187 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:30:58,187 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:30:58,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:30:58,437 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,437 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 09:30:58,450 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:58,479 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:30:58,506 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:30:58,506 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,506 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 09:30:58,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:30:58,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,696 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 09:30:58,757 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:30:58,757 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,758 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,766 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,826 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:30:58,826 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,826 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 09:30:58,826 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=null 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,828 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 09:30:58,830 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:58,830 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:58,831 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:30:58,831 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 09:30:58,831 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:58,832 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 09:30:58,832 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:30:58,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:30:58,832 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:30:58,832 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 09:30:58,832 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:58,849 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:30:58,892 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] 09:30:58,892 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 09:30:58,892 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191] 09:30:58,947 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 09:30:58,976 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:30:58,976 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,976 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 09:30:58,977 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,978 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 09:30:58,979 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:30:58,979 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:30:58,980 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 09:30:58,980 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:58,980 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 09:30:58,980 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:30:58,980 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:30:58,980 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:30:58,980 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:30:58,981 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 09:30:58,981 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:30:58,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] 09:30:58,994 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:30:58,994 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:58,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:58,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] 09:30:58,994 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:30:58,994 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] 09:30:58,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] 09:30:58,995 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionForwardedToLeaderAfterRetry commit rateLimit adjusted to 79.76650015107607 09:30:58,995 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:58,995 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] 09:30:58,995 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 09:30:58,995 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] 09:30:58,995 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 09:30:58,995 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying 2 components for commit 09:30:58,995 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard cars 09:30:58,995 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 09:30:58,995 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Readying transaction for shard people 09:30:58,995 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Queuing TransactionOperation 09:30:58,995 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 canCommit 09:30:58,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:30:58,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:30:58,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:30:58,995 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:30:58,995 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 = [ 1 ] 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction called 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:30:58,996 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, disabling transaction tracking 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:59,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,1 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 09:30:59,2 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 12 09:30:59,2 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,2 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 09:30:59,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 of 1 operations, client version 12 09:30:59,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,2 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 ready 09:30:59,2 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 09:30:59,3 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}]]] 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 09:30:59,3 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] resolved 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 readyTransaction 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] resolved 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: successfully resolved all cohort actors 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finishCanCommit 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,3 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,4 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69}] 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=3419069} is already disabled 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:59,4 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=3665064} is already disabled 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:30:59,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, size=69} 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} does not match any client, pre-disabling client. 09:30:59,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:30:59,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:30:59,6 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:59,6 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,6 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,7 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,7 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,7 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 validated 09:30:59,7 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:59,7 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: success 09:30:59,7 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:59,7 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit returning result: true 09:30:59,7 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 commit 09:30:59,8 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 finish commit 09:30:59,8 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,8 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,8 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,8 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 candidate ready 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 09:30:59,9 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: canCommit - messages: [] 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 prepared 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Finishing commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:30:59,9 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:30:59,10 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:30:59,10 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:30:59,10 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 09:30:59,10 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 submitted to persistence 09:30:59,11 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=179} 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=183} 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:30:59,11 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:30:59,11 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry%23-309517191$f] 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry%23412688246$e] 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1803763019] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:30:59,12 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1803763019] 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1643211734] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:30:59,12 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 3 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:30:59,12 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:30:59,12 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:30:59,12 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1803763019] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:59,13 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1643211734] 09:30:59,13 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 272.5 μs 09:30:59,13 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:30:59,13 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1643211734] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:59,13 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 59.77 μs 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=183} 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data Simple{size=179} 09:30:59,14 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,14 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:30:59,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#102134410] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:30:59,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:30:59,15 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1869387019] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:30:59,15 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:30:59,15 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 3 09:30:59,15 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0: commit succeeded 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1869387019] 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 84.59 μs 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1869387019] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#102134410] 09:30:59,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:30:59,16 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 30.82 μs 09:30:59,16 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#102134410] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:30:59,17 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] 09:30:59,17 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:30:59,18 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] 09:30:59,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#1197791238] 09:30:59,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#1080864564] 09:30:59,18 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 09:30:59,18 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1012721072] 09:30:59,18 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-1012721072] 09:30:59,18 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,18 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:59,18 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:59,18 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:59,18 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:30:59,19 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 09:30:59,19 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:30:59,19 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying 2 components for commit 09:30:59,19 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard cars 09:30:59,19 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 09:30:59,19 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 09:30:59,19 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 Readying transaction for shard people 09:30:59,19 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 = [ 1 ] 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction called 09:30:59,19 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 sending 1 batched modifications, ready: true 09:30:59,19 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 09:30:59,19 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 canCommit 09:30:59,20 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 12 09:30:59,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,20 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 09:30:59,20 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 of 1 operations, client version 12 09:30:59,20 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,20 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 ready 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] resolved 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 readyTransaction 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] resolved 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: successfully resolved all cohort actors 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 finishCanCommit 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246)] 09:30:59,21 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 09:30:59,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:59,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 09:30:59,30 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:59,30 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: sending CanCommitTransaction [transactionId=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,30 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-309517191)] 09:30:59,31 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Can committing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,31 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:30:59,31 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 validated 09:30:59,31 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:30:59,31 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: canCommit for member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: success 09:30:59,32 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:30:59,32 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0: canCommit returning result: true 09:30:59,32 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:30:59,33 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] for shard cars, trying to use DirectTransactionContextWrapper 09:30:59,33 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 09:30:59,33 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] for shard people, trying to use DirectTransactionContextWrapper 09:30:59,33 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] found - creating WRITE_ONLY transaction context 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person 09:30:59,33 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying 2 components for commit 09:30:59,33 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard cars 09:30:59,33 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 09:30:59,33 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry)] 09:30:59,33 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 Readying transaction for shard people 09:30:59,33 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 = [ 1 ] 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction called 09:30:59,33 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 sending 1 batched modifications, ready: true 09:30:59,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry)] 09:30:59,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] 09:30:59,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:30:59,34 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,34 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,35 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:30:59,35 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 12 09:30:59,35 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:30:59,35 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 09:30:59,35 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying 1 batched modifications for Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:30:59,36 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Readying Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 of 1 operations, client version 12 09:30:59,36 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:30:59,36 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 ready 09:30:59,37 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 09:30:59,37 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,37 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 readyTransaction 09:30:59,37 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 modifications completed with ReadyTransactionReply [version=12] 09:30:59,86 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,136 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,187 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,218 AM [cluster-test-shard-dispatcher-20] [WARN] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 200, lastApplied : 1, commitIndex : 1 09:30:59,237 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,287 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,337 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,388 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,438 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,488 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,538 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,589 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,639 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,689 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,740 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,777 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:30:59,790 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,841 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,891 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,941 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:30:59,992 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,42 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,93 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,143 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,194 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,244 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,295 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,345 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,402 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,453 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,504 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,554 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,604 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,655 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,705 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,755 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,806 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,856 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,907 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:00,958 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,8 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,58 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,109 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,159 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,210 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,260 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,311 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,361 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,411 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,461 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,512 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,612 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,663 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,713 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,764 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,814 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,865 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,915 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:01,965 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,117 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,167 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,217 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,268 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,318 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,368 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,419 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,469 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,519 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,570 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,620 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,671 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,721 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,772 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,822 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,873 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,923 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:02,974 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,24 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,175 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,225 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,326 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,377 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,478 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,528 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,578 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,629 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,679 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,729 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,780 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,830 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,881 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,931 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:03,982 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,32 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,82 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,133 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,183 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,234 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,284 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,335 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,385 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,436 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,486 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,537 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,587 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,638 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,688 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,739 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,789 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,840 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,892 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,943 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:04,994 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246] 09:31:05,44 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 09:31:05,45 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:31:05,45 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:31:05,45 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:31:05,45 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:31:05,45 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:31:05,46 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 09:31:05,46 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:31:05,50 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:31:05,51 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:31:05,51 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 09:31:05,51 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:31:05,51 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#999140591] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#999140591] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:05,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:31:05,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:31:05,55 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 09:31:05,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:31:05,55 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 09:31:05,56 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b7f77b2 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:31:05,56 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 09:31:05,56 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:05,56 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@13a10d3a 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:05,56 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:05,266 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 09:31:05,266 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:05,266 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 09:31:05,266 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:05,466 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 09:31:05,466 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#412688246]] after [400 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:31:05,466 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Failed to transfer leadership in 410.8 ms 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Failed to transfer leadership in 411.1 ms 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: In unpauseLeader 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: In unpauseLeader 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: leader transfer failed - sending PoisonPill 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: leader transfer failed - sending PoisonPill 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-705: Closing 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-704: Closing 09:31:05,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 2 pending queued transactions 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 2 pending queued transactions 09:31:05,467 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:31:05,468 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:31:05,469 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 09:31:05,469 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:31:05,469 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-734010205] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:05,471 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:05,471 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:05,480 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:05,480 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:05,480 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:05,480 AM [cluster-test-akka.actor.default-dispatcher-21] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:05,482 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:05,487 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:05,487 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:06,489 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2249789142914040770] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:06,501 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:06,501 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:06,503 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:06.504] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:06,510 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7869945781402327951] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:06,512 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:06,512 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:06,514 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:06.515] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:06,536 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:06,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [6375870353040583991] 09:31:06,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:06,542 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:06,542 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:06,553 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:06,553 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:06,555 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:06,555 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:06,555 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:06,560 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:06,568 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-4362771533124548619] 09:31:06,568 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:06,573 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:06,573 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:06,573 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:06,573 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:06,598 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:06,606 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7134453300596712523] 09:31:06,607 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:06,619 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:06,619 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:06,620 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:06,620 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:06,628 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:06,630 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:06,631 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,639 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,645 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2088232956] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,650 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:31:06,651 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:06,651 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1ce49e0d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d750108 09:31:06,655 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2c35988e 09:31:06,655 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037466651, None)) 09:31:06,657 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,657 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037466650,0,0)) 09:31:06,659 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d750108 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b3cf795 09:31:06,659 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:31:06,660 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:06,661 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:06,662 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,665 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:06,665 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,665 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3b88c951 09:31:06,665 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:06,665 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,666 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,666 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,666 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1852747909] 09:31:06,666 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:31:06,666 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1852747909] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,667 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1245082213 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,667 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1245082213] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,667 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-439594978] 09:31:06,667 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:31:06,667 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-439594978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:06,668 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:31:06,668 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:31:06,668 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,671 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:31:06,672 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,673 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,680 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:31:06,680 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:06,680 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@64f7c8cb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4a2089da 09:31:06,680 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@47ea2dbf 09:31:06,680 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,681 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037466682, None)) 09:31:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037466681,0,0)) 09:31:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4a2089da to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@695a5791 09:31:06,682 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:31:06,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@60b45b58 09:31:06,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:06,683 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,687 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:06,691 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:31:06,691 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:31:06,692 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1686831720] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,692 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#260772808 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,693 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#260772808] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,693 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#1936415884] 09:31:06,693 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#1936415884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:06,695 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:06,696 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:06,696 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,696 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,697 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:06,698 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:06,698 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:06,698 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:31:06,698 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:31:06,698 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:31:06,698 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:06,698 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:31:06,698 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,698 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,699 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:06,701 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:31:06,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2088232956] 09:31:06,733 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 09:31:06,733 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1852747909] 09:31:06,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:06,914 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:06,915 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 09:31:06,915 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 09:31:07,194 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:31:07,194 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,194 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 09:31:07,247 AM [cluster-test-shard-dispatcher-17] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:07,495 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:31:07,495 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,495 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 09:31:07,565 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:07,565 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,566 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,566 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,566 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,602 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,602 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,635 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:07,804 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:31:07,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,805 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 09:31:07,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,805 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1852747909] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:07,808 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:07,808 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:07,808 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:31:07,808 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:07,808 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 09:31:07,809 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 09:31:07,809 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:31:07,810 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1852747909] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:07,816 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:31:07,817 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 09:31:07,817 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:07,839 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:07,841 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:07,844 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,846 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-1166854437] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,847 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:31:07,849 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:07,849 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5ff34dd0 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f42fcb 09:31:07,849 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2406e072 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:07,849 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,849 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037467849, None)) 09:31:07,850 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037467848,0,0)) 09:31:07,850 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@96a934 09:31:07,850 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2f42fcb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6947643b 09:31:07,850 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:31:07,850 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 09:31:07,851 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] 09:31:07,851 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Queuing TransactionOperation 09:31:07,851 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:07,851 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=12] 09:31:07,851 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=12] 09:31:07,851 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars 09:31:07,851 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 09:31:07,851 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,852 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] 09:31:07,852 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:31:07,853 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:31:07,854 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, disabling transaction tracking 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,854 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-219688944 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:07,854 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-219688944] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,854 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,855 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 09:31:07,855 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 09:31:07,855 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_12#-1320254812] 09:31:07,855 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_12#-1320254812] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,856 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#492449401] 09:31:07,856 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#492449401] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 09:31:07,856 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:07,857 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_12#-1320254812, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0, version=12] 09:31:07,857 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:07,857 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_12#-1320254812)] 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=3, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 09:31:07,857 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:31:07,857 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:07,857 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,858 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,858 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 09:31:07,859 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:31:07,859 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,859 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,859 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 09:31:07,859 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 09:31:07,859 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,859 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:07,859 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 09:31:07,859 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 09:31:07,859 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:31:07,859 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:07,860 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 ReadData operation succeeded 09:31:07,862 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:07,862 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,862 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-710: Closing 09:31:07,862 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:31:07,862 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,862 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:07,862 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]]] 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 09:31:07,863 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:07,864 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}] 09:31:07,865 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 09:31:07,868 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:07,868 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:07,868 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:07,868 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:07,869 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:07,887 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,888 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionRetryWithInitialAskTimeoutExOnCreateTx commit rateLimit adjusted to 79.76650015107607 09:31:07,888 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:07,888 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:07,888 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:07,888 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 09:31:07,888 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 09:31:07,888 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 09:31:07,888 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying 1 components for commit 09:31:07,888 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Readying transaction for shard cars 09:31:07,888 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Queuing TransactionOperation 09:31:07,888 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 canCommit 09:31:07,889 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:31:07,889 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:31:07,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:07,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:31:07,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 09:31:07,889 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:08,406 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:08,436 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:08,437 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] found - trying create transaction 09:31:08,437 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)] 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 09:31:08,447 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:08,448 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:08,448 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 09:31:08,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:08,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 09:31:08,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:08,449 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:31:08,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 09:31:08,450 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 0, lastTerm: 4 09:31:08,450 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,450 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:31:08,451 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:31:08,451 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:31:08,451 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): LastApplied index -1 is behind last index 0 - switching to PreLeader 09:31:08,456 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:31:08,456 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:08,456 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:08,457 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 5 09:31:08,457 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Replicate message: identifier: null, logIndex: 1, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,458 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to PreLeader 09:31:08,458 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:31:08,459 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:08,459 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:08,459 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to PreLeader 09:31:08,459 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:08,459 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:08,459 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,460 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:31:08,460 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:08,460 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:08,460 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:31:08,460 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 09:31:08,460 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 09:31:08,460 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 09:31:08,460 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,460 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,461 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:08,464 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=4, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 2 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:08,464 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 1, nextIndex: 2 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Not updating commit index to 0 - retrieved log entry with index 0, term 4 does not match the current term 5 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 0 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader state for ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} not found 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Setting last applied to 1 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:08,465 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader): Received ApplyState [identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, replicatedLogEntry=SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80}]] - lastApplied: 1, lastIndex: 1 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 5 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (PreLeader)_slicer-id-713: Closing 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 09:31:08,466 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 09:31:08,466 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from PreLeader to Leader 09:31:08,466 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from PreLeader to Leader 09:31:08,466 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 1 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 0 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, size=80} 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} does not match any client, pre-disabling client. 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 1 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 0 to 1 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 1 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:08,467 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=1 09:31:08,875 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6375870353040583991] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:08,876 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [6375870353040583991] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:08,956 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx)]] after [500 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:08,986 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:08,986 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] 09:31:08,986 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:08,987 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1686831720, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@593d3767] 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1686831720, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@593d3767] 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1686831720)] found - trying create transaction 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1686831720)] 09:31:08,987 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating transaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_13#-391656916] 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_13#-391656916, transactionId=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, version=12] 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 directCommit called 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_13#-391656916] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 sending 1 batched modifications, ready: true 09:31:08,987 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:1_13#-391656916)] 09:31:08,988 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: handleForwardedReadyTransaction for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, client version 12 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: readying transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 ready 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 validated 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: canCommit for member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: success 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 candidate ready 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - candidate rootPath: / 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: canCommit - messages: [] 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 prepared 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Finishing commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 191 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 09:31:08,988 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 submitted to persistence 09:31:08,989 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}]]] 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=191} 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 3 09:31:08,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:08,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=191}] 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 2, nextIndex: 3 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 2 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 1 to 2 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 2 data Simple{size=191} 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0: commit - successfulFromPrevious: [] 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_shard-cars-member-2%3Adatastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx%400%3A1_13%23-391656916$c] 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1957920661] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:08,997 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=2 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 modifications completed with CommitTransactionReply [version=12] 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 successfully completed direct commit 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1957920661] 09:31:08,998 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1957920661] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:08,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 281.4 μs 09:31:08,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Commit index set to 2 09:31:08,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:31:08,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Setting last applied to 2 09:31:08,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from 1 to 2 09:31:08,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 2 data Simple{size=191} 09:31:08,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying foreign transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:31:08,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1434722626] with name member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 09:31:08,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1434722626] 09:31:08,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=2 09:31:08,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 09:31:08,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:08,999 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 09:31:08,999 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 36.56 μs 09:31:08,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 09:31:08,999 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1434722626] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,0 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:09,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:31:09,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessagesSuccessful$] from Actor[akka://cluster-test/system/in-memory-journal#-495333239] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessageSuccess] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#181295951] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 09:31:09,0 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:09,0 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:09,0 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:09,0 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#645510908] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#645510908] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,1 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:09,1 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initiating leader transfer 09:31:09,1 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=12] 09:31:09,1 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:09,1 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c4e2d1c 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5c4e2d1c 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Attempting to transfer leadership 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:09,1 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:09,537 AM [cluster-test-shard-dispatcher-20] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Failed to transfer leadership in 535.9 ms 09:31:09,537 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In unpauseLeader 09:31:09,537 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader transfer failed - sending PoisonPill 09:31:09,537 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Leadership transfer expired 09:31:09,537 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:09,538 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-714: Closing 09:31:09,538 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:31:09,538 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 09:31:09,538 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:31:09,539 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:09,539 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:31:09,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 09:31:09,539 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:09,539 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,539 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:31:09,540 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:31:09,539 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-12228435] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-12228435] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,540 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#978843988] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:09,540 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:09,540 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:10,555 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-4362771533124548619] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:10,558 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:10,558 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:10,559 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:10.561] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:10,578 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:10,578 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:10,580 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:10.582] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:10,610 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:10,621 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-954185658850737766] 09:31:10,621 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:10,622 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:10,622 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:10,633 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:10,633 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:10,633 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:10,633 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:10,633 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:10,655 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:31:10,672 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [7530764778874642351] 09:31:10,672 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:10,679 AM [cluster-test-akka.actor.default-dispatcher-6] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:10,679 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:10,681 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:10,681 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:10,720 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:10,771 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:10,773 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:10,788 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-66782871820681822] 09:31:10,788 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:10,801 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:10,802 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:10,804 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,805 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:10,805 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:10,805 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:10,805 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:10,808 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1848830134] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,808 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,811 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:31:10,811 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:10,812 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5ab95a6a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@10d31d46 09:31:10,812 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5bf1d658 09:31:10,812 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1714037470812, None)) 09:31:10,812 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037470811,0,0)) 09:31:10,812 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@10d31d46 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@e0ee062 09:31:10,812 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@34e0ba78 09:31:10,812 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 09:31:10,812 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:10,812 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,812 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:10,813 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:10,815 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551] 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051] 09:31:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 09:31:10,816 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,817 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:31:10,817 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 09:31:10,817 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,817 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-86306187 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,819 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1783149026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,819 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-86306187] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:10,822 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:10,823 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1783149026 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,824 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1970475249] 09:31:10,824 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:31:10,824 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1970475249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 09:31:10,824 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 09:31:10,824 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:10,824 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:10,824 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:10,824 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:10,824 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 09:31:10,824 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:10,824 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 09:31:10,824 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:10,824 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,825 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:10,825 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 09:31:10,825 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:10,825 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,825 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 09:31:10,828 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:31:10,828 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1321160945] 09:31:10,828 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1321160945] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,837 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2051470935] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:31:10,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:10,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@44685bb3 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3be685d8 09:31:10,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2c95894b 09:31:10,847 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 09:31:10,847 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1714037470847, None)) 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,847 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,848 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037470846,0,0)) 09:31:10,848 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3be685d8 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1afc1471 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4834db63 09:31:10,848 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#599590872] 09:31:10,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,849 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 09:31:10,849 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:31:10,849 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 09:31:10,850 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:31:10,850 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:10,850 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#599590872] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,850 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1411299600] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,851 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1107451231] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,851 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-388458954 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,851 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#-388458954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,851 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-411144670] 09:31:10,851 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-411144670] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,857 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,857 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,857 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#-1411299600] 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 09:31:10,858 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 09:31:10,858 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:10,859 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:10,859 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:10,859 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:10,859 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:10,859 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:10,862 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1105381528] 09:31:10,862 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1105381528] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:10,862 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1107451231 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:10,862 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:10,862 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,864 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 09:31:10,864 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 09:31:10,864 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 09:31:10,865 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:31:10,865 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 09:31:10,898 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1848830134] 09:31:10,898 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 09:31:10,899 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551] 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:11,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:11,59 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:31:11,60 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 09:31:11,88 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:31:11,89 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:31:11,89 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 09:31:11,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:11,319 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,319 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:31:11,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:11,378 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,378 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:31:11,424 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:11,434 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:11,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:11,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,549 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:31:11,650 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:11,650 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,650 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,659 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:11,659 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,659 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:31:11,659 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,659 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:11,659 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,659 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,660 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 3 in "RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=null 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=3, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,661 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=3, voteGranted=true] 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=true], my term is 3 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=3, voteGranted=true], current voteCount: 1 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 3 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:11,662 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:11,662 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:11,663 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:11,663 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:11,663 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:11,756 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#1848830134] 09:31:11,757 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 09:31:11,757 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051] 09:31:11,781 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 09:31:11,808 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:11,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:11,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,859 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:31:11,859 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,860 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:11,860 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:31:11,860 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:11,860 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:11,860 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:31:11,861 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:11,861 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,861 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:31:11,862 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 09:31:11,862 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 09:31:11,863 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 09:31:11,863 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,863 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:11,863 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:11,863 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:11,863 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 09:31:11,863 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 09:31:11,864 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:11,864 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:11,908 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2051470935] 09:31:11,908 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:11,908 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:11,908 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2051470935] 09:31:11,908 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:11,908 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:11,908 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 09:31:11,908 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:31:11,908 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 09:31:11,908 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:11,908 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:11,908 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 09:31:11,908 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:11,908 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:11,908 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 09:31:11,910 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 09:31:11,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,911 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_14#1405687547] 09:31:11,911 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_14#1405687547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,911 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_15#402460620] 09:31:11,911 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_15#402460620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_14#1405687547, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}]]] 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:31:11,912 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_14#1405687547)] 09:31:11,912 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_15#402460620, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] 09:31:11,912 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:31:11,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 09:31:11,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 09:31:11,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:31:11,913 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:0_15#402460620)] 09:31:11,913 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,913 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,913 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=3, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=16477533} is already disabled 09:31:11,926 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:11,926 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 12 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,926 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69}] 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=16740997} is already disabled 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: handleForwardedReadyTransaction for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, client version 12 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,927 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 09:31:11,928 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 closeTransaction called 09:31:11,928 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-cars-member-2:datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx@0:0_12#-1320254812)] [ERROR] [04/25/2024 09:31:11.929] [com.google.common.base.internal.Finalizer] [akka://cluster-test@127.0.0.1:2558/] swallowing exception during message send java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.remote.artery.Association.setupStopQuarantinedTimer(Association.scala:633) at akka.remote.artery.Association.send(Association.scala:399) at akka.remote.artery.ArteryTransport.send(ArteryTransport.scala:707) at akka.remote.RemoteActorRef.$bang(RemoteActorRefProvider.scala:737) at akka.actor.ActorRef.tell(ActorRef.scala:131) at akka.actor.ActorSelection$.rec$1(ActorSelection.scala:287) at akka.actor.ActorSelection$.deliverSelection(ActorSelection.scala:291) at akka.actor.ActorSelection.tell(ActorSelection.scala:50) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.sendOperationAsync(ActorUtils.java:417) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContext.closeTransaction(RemoteTransactionContext.java:84) at org.opendaylight.controller.cluster.datastore.TransactionContextCleanup.finalizeReferent(TransactionContextCleanup.java:60) at jdk.internal.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at com.google.common.base.internal.Finalizer.cleanUp(Finalizer.java:181) at com.google.common.base.internal.Finalizer.run(Finalizer.java:146) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:31:11,930 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 09:31:11,930 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551)] resolved 09:31:11,930 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051)] resolved 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551)] 09:31:11,931 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551)] 09:31:11,932 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,932 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,932 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 09:31:11,932 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:11,932 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 09:31:11,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:31:11,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051)] 09:31:11,933 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051)] 09:31:11,934 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,934 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,934 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 09:31:11,934 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:11,934 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 09:31:11,934 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:31:11,934 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 09:31:11,934 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 09:31:11,934 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 09:31:11,935 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551)] 09:31:11,935 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-624578551)] 09:31:11,935 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051)] 09:31:11,935 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#-1684595051)] 09:31:11,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,935 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 09:31:11,935 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 09:31:11,936 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:31:11,936 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=3, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}]]] 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}]]] 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=183} 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 2 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=3, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=3, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:31:11,938 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:31:11,938 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, size=69} 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=183}] 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=3, payload=Simple{size=179}] 09:31:11,939 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:11,939 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-cars-testReadWriteTransactionWithMultipleShards%23-624578551$g] 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testReadWriteTransactionWithMultipleShards_member-1-shard-people-testReadWriteTransactionWithMultipleShards%23-1684595051$h] 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#825075708] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1607661822] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:11,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:11,940 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:11,941 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1607661822] 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#825075708] 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 59.02 μs 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 89.01 μs 09:31:11,941 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1607661822] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,943 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 09:31:11,943 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 1 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 09:31:11,943 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=183} 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:11,943 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 09:31:11,943 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:11,943 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards)] 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:11,943 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#490696590] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:11,943 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1935960355] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:31:11,943 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1935960355] 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 3 09:31:11,943 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:11,943 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:11,943 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 37.81 μs 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1935960355] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#490696590] 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 27.35 μs 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#490696590] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:11,944 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_16#1039198389] 09:31:11,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_16#1039198389, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=12] 09:31:11,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:11,945 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_16#1039198389)] 09:31:11,948 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 09:31:11,948 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 09:31:11,948 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:11,948 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:11,948 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards)] 09:31:11,948 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 09:31:11,949 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:11,949 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_17#246721381] 09:31:11,951 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_17#246721381, transactionId=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=12] 09:31:11,951 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:31:11,951 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_17#246721381)] 09:31:11,953 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 09:31:11,953 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,954 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,954 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 09:31:11,954 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 09:31:11,954 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 09:31:11,954 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,954 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:31:11,954 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:11,954 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,954 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:31:11,954 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:11,954 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:11,954 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:31:11,955 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:31:11,955 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64e08c0e 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@64e08c0e 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:11,955 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2051470935] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#2051470935] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:11,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 09:31:11,955 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c387543 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6c387543 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:11,955 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:11,955 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:12,218 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Failed to transfer leadership in 263.1 ms 09:31:12,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In unpauseLeader 09:31:12,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:12,218 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:31:12,219 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:31:12,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-722: Closing 09:31:12,219 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Failed to transfer leadership in 303.5 ms 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In unpauseLeader 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-721: Closing 09:31:12,259 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:12,260 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:12,260 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:31:12,260 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:31:12,260 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 09:31:12,260 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:31:12,261 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:12,261 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:12,265 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:12,265 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:12,265 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:12,265 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:12,281 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:12,281 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:13,278 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-954185658850737766] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:13,298 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:13,298 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:13,303 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [7530764778874642351] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:13,304 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:13.307] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:13,315 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:13,315 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:13,318 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:13.319] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:13,335 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:13,340 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1664281533844494692] 09:31:13,340 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:13,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:13,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:13,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:13,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:13,351 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:13,351 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:13,351 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:13,355 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:13,366 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-43974267774206643] 09:31:13,366 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:13,368 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:13,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:13,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:13,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:13,389 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:13,396 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1276431528891353104] 09:31:13,396 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:13,405 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:13,405 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:13,410 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:13,410 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:13,420 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:13,420 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:13,434 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:13,436 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:13,436 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:13,438 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:13,444 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:13,445 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:14,454 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:14.456] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:14,464 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:14,464 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:14,466 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:14.467] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:14,483 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:14,490 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5235723619826004088] 09:31:14,490 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:14,492 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:14,492 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:14,502 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:14,502 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:14,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:14,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:14,504 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:14,510 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:14,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3790287169391222277] 09:31:14,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:14,524 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:14,524 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:14,524 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:14,524 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:14,535 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:14,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8645412400471985380] 09:31:14,547 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:14,559 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:14,559 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:14,560 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:14,562 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:14,563 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 09:31:14,564 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:14,564 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:14,570 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1659291375] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,574 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:14,582 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 09:31:14,582 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:14,582 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6d03afbd to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43bdb691 09:31:14,582 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@26f34312 09:31:14,582 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:14,582 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,583 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:14,583 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,584 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1714037474583, None)) 09:31:14,584 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037474582,0,0)) 09:31:14,585 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43bdb691 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@17eecb45 09:31:14,585 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 09:31:14,585 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52fd6d79 09:31:14,585 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:14,585 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,585 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:14,586 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,586 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,586 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] 09:31:14,586 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 09:31:14,587 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 09:31:14,588 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:14,590 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 09:31:14,591 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,591 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1684473081] 09:31:14,592 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-1412656930 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,592 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#-1412656930] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,592 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#1684473081] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,592 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:14,595 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:14,595 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:14,596 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:31:14,596 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:31:14,596 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 09:31:14,596 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 09:31:14,596 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 09:31:14,596 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 09:31:14,596 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 09:31:14,596 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:14,604 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,614 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:14,614 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:14,620 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:14,624 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 09:31:14,624 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:14,625 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@209ddcfd to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9e48334 09:31:14,625 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4ce60c6c 09:31:14,625 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1714037474625, None)) 09:31:14,625 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:14,625 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,625 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037474624,0,0)) 09:31:14,625 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:14,625 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,625 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@32c559a3 09:31:14,625 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@9e48334 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5184cd09 09:31:14,625 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 09:31:14,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:14,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,626 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,626 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#2054888501] 09:31:14,627 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 09:31:14,627 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#2054888501] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1621203171 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,627 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1621203171] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,627 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-411248261] 09:31:14,627 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 09:31:14,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#-411248261] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:14,627 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:14,627 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:14,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 09:31:14,627 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 09:31:14,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:31:14,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:31:14,628 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 09:31:14,628 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,628 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 09:31:14,628 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 09:31:14,676 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1659291375] 09:31:14,676 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 09:31:14,676 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 09:31:14,860 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:31:14,861 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:31:14,861 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 09:31:14,861 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 09:31:15,150 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:31:15,150 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,150 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 09:31:15,174 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:15,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:31:15,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,430 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 09:31:15,523 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:15,523 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,524 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,531 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:15,532 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,532 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,532 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:15,532 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,532 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,570 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:15,690 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:31:15,690 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,690 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 09:31:15,691 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:15,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 09:31:15,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:15,693 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:15,693 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 09:31:15,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:15,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,694 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:15,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:15,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:15,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 09:31:15,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 09:31:15,696 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=12] 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 09:31:15,696 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 09:31:15,700 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 09:31:15,700 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,700 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 09:31:15,700 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=12] 09:31:15,701 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:15,701 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:15,701 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 09:31:15,701 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 09:31:15,701 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:15,733 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,733 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] 09:31:15,733 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:15,733 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:15,734 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying 1 components for commit 09:31:15,734 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 09:31:15,734 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:15,734 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 canCommit 09:31:15,734 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:15,734 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=12] 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=12] 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 directCommit called 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 sending 2 batched modifications, ready: true 09:31:15,734 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, disabling transaction tracking 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,738 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@a43601c 09:31:15,739 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 2 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 of 2 operations, client version 12 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 ready 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 validated 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: success 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 candidate ready 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: canCommit - messages: [] 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 prepared 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 183 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 09:31:15,740 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 09:31:15,740 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}]]] 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 submitted to persistence 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 1 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 09:31:15,741 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 09:31:15,741 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72}] 09:31:15,742 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 0, nextIndex: 1 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=4179991} is already disabled 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 09:31:15,742 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}]]] 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=183} 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 0 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, size=72} 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:15,743 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} does not match any client, pre-disabling client. 09:31:15,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74}] 09:31:15,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=183}] 09:31:15,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=0 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 2, nextIndex: 3 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} has no callback 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$a] 09:31:15,751 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-214700429] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 09:31:15,752 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 09:31:15,752 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:15,752 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 09:31:15,752 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-214700429] 09:31:15,752 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,752 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 107.4 μs 09:31:15,752 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-214700429] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,753 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 successfully completed direct commit 09:31:15,754 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,754 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,754 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,754 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,754 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,754 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}] 09:31:15,755 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying 1 components for commit 09:31:15,755 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 Readying transaction for shard cars 09:31:15,755 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 directCommit called 09:31:15,755 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 sending 1 batched modifications, ready: true 09:31:15,755 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,755 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 canCommit 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 2 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0}, size=74} 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data Simple{size=183} 09:31:15,755 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:31:15,756 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1565197602] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 09:31:15,756 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1565197602] 09:31:15,756 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,756 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 1 for term 4 09:31:15,756 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 41.22 μs 09:31:15,756 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#-1565197602] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,756 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=2 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@33344acc 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 of 1 operations, client version 12 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,757 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 ready 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 validated 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: success 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 candidate ready 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - candidate rootPath: / 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: canCommit - messages: [] 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: preCommit - successfulFromPrevious: [] 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 prepared 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:31:15,758 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,759 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 09:31:15,759 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 submitted to persistence 09:31:15,759 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}]]] 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,760 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=240}] 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0: commit - successfulFromPrevious: [] 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$b] 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:15,761 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 09:31:15,761 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,761 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 62.02 μs 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 09:31:15,763 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 successfully completed direct commit 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=240} 09:31:15,763 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,763 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:31:15,764 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 09:31:15,764 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 09:31:15,764 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 63.19 μs 09:31:15,764 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 09:31:15,764 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,764 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,764 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:15,764 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,764 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 09:31:15,765 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 09:31:15,765 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,765 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6f85e207 09:31:15,766 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_18#451002987] 09:31:15,766 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_18#451002987] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,766 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_18#451002987, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0, version=12] 09:31:15,767 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,767 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_18#451002987)] 09:31:15,769 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 ReadData operation succeeded 09:31:15,769 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 closeTransaction called 09:31:15,769 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-2_18#451002987)] 09:31:15,769 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,769 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,769 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,769 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,769 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}] 09:31:15,769 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying 1 components for commit 09:31:15,770 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 Readying transaction for shard cars 09:31:15,770 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 directCommit called 09:31:15,770 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 sending 1 batched modifications, ready: true 09:31:15,770 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,770 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 canCommit 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@6f85e207 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 of 1 operations, client version 12 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 ready 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 validated 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: success 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 candidate ready 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - candidate rootPath: / 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: canCommit - messages: [] 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: preCommit - successfulFromPrevious: [] 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 prepared 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,771 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 submitted to persistence 09:31:15,772 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 09:31:15,772 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}]]] 09:31:15,772 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,773 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=240}] 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0: commit - successfulFromPrevious: [] 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:15,773 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$e] 09:31:15,774 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 09:31:15,774 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:31:15,774 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 09:31:15,774 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,774 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 152.2 μs 09:31:15,776 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 successfully completed direct commit 09:31:15,776 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,776 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 09:31:15,776 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,776 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,776 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:15,776 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,776 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Queuing TransactionOperation 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data Simple{size=240} 09:31:15,776 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:31:15,777 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,777 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 72.43 μs 09:31:15,777 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 09:31:15,777 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 09:31:15,777 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 09:31:15,777 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,777 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1ab58c75 09:31:15,778 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_19#46161482] 09:31:15,778 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_19#46161482] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,778 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_19#46161482, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0, version=12] 09:31:15,778 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,778 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_19#46161482)] 09:31:15,780 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 ReadData operation succeeded 09:31:15,780 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 closeTransaction called 09:31:15,780 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-4_19#46161482)] 09:31:15,781 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,781 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,781 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,781 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,781 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}] 09:31:15,781 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying 1 components for commit 09:31:15,781 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 Readying transaction for shard cars 09:31:15,781 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 directCommit called 09:31:15,781 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 sending 1 batched modifications, ready: true 09:31:15,781 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,781 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 canCommit 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1ab58c75 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 of 1 operations, client version 12 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 ready 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 validated 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: success 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,783 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 candidate ready 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - candidate rootPath: / 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: canCommit - messages: [] 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: preCommit - successfulFromPrevious: [] 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 prepared 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 3 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 09:31:15,784 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 submitted to persistence 09:31:15,785 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}]]] 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:31:15,785 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,786 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=240}] 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 5, nextIndex: 6 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0: commit - successfulFromPrevious: [] 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$h] 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:31:15,786 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 09:31:15,786 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,787 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 36.31 μs 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 5 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 09:31:15,787 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 successfully completed direct commit 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data Simple{size=240} 09:31:15,787 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,787 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:31:15,788 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,788 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 4 for term 4 09:31:15,788 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 34.07 μs 09:31:15,788 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=5 09:31:15,788 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 09:31:15,788 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,788 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,788 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:15,788 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,788 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Queuing TransactionOperation 09:31:15,789 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 09:31:15,789 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,789 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@606d64a9 09:31:15,789 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_20#769958893] 09:31:15,789 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_20#769958893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,789 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_20#769958893, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0, version=12] 09:31:15,790 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,790 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_20#769958893)] 09:31:15,793 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 ReadData operation succeeded 09:31:15,793 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 closeTransaction called 09:31:15,793 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-6_20#769958893)] 09:31:15,794 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,794 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,794 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,794 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,794 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}] 09:31:15,794 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying 1 components for commit 09:31:15,794 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 Readying transaction for shard cars 09:31:15,794 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 directCommit called 09:31:15,794 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 sending 1 batched modifications, ready: true 09:31:15,794 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,794 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 canCommit 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@606d64a9 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 of 1 operations, client version 12 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 ready 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 validated 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: success 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 candidate ready 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - candidate rootPath: / 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: canCommit - messages: [] 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: preCommit - successfulFromPrevious: [] 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 prepared 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 6, leaderLastIndex: 6, leaderSnapShotIndex: 4 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 6 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 09:31:15,796 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 submitted to persistence 09:31:15,797 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}]]] 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 5, entries to be added from: 0 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:31:15,797 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,798 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=240}] 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 5 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0: commit - successfulFromPrevious: [] 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$k] 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 09:31:15,798 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 09:31:15,799 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 09:31:15,799 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,799 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 38.77 μs 09:31:15,799 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,799 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 successfully completed direct commit 09:31:15,799 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 09:31:15,799 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,799 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,799 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:15,799 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,799 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Queuing TransactionOperation 09:31:15,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 09:31:15,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 5, lastIndex(): 6 09:31:15,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 09:31:15,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 09:31:15,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=240} 09:31:15,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:31:15,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 09:31:15,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 09:31:15,800 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,800 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 91.29 μs 09:31:15,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 09:31:15,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1b68488d 09:31:15,801 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_21#21948083] 09:31:15,801 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_21#21948083] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,802 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_21#21948083, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0, version=12] 09:31:15,802 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,802 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_21#21948083)] 09:31:15,804 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 ReadData operation succeeded 09:31:15,804 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 closeTransaction called 09:31:15,804 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-8_21#21948083)] 09:31:15,804 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,804 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,804 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - creating WRITE_ONLY transaction context 09:31:15,804 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}] 09:31:15,804 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying 1 components for commit 09:31:15,805 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 Readying transaction for shard cars 09:31:15,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 directCommit called 09:31:15,805 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 sending 1 batched modifications, ready: true 09:31:15,805 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,805 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 canCommit 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1b68488d 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying 1 batched modifications for Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Readying Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 of 1 operations, client version 12 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 ready 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 validated 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: canCommit for member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: success 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 candidate ready 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - candidate rootPath: / 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: canCommit - messages: [] 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: preCommit - successfulFromPrevious: [] 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 prepared 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Finishing commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 09:31:15,807 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:15,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 7, leaderSnapShotIndex: 5 09:31:15,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:15,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 09:31:15,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 submitted to persistence 09:31:15,809 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 09:31:15,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}]]] 09:31:15,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:31:15,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:31:15,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=240} 09:31:15,809 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 09:31:15,810 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:31:15,810 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=7, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:15,810 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=Simple{size=240}] 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 7, nextIndex: 8 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 7, lastAppliedIndex: 6 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0: commit - successfulFromPrevious: [] 09:31:15,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 changing state from COMMIT_PENDING to COMMITTED 09:31:15,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:15,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleTransactionsWritesInQuickSuccession_member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession$n] 09:31:15,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 09:31:15,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 09:31:15,811 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 09:31:15,811 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,811 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 91.88 μs 09:31:15,812 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 successfully completed direct commit 09:31:15,812 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 modifications completed with CommitTransactionReply [version=12] 09:31:15,812 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 09:31:15,812 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:15,812 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:15,812 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:15,812 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:15,813 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Queuing TransactionOperation 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 7 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 7, context.getLastApplied(): 6, lastIndex(): 7 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data Simple{size=240} 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 6 for term 4 09:31:15,813 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:31:15,813 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=7 09:31:15,813 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 35.56 μs 09:31:15,814 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 09:31:15,814 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:15,814 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1eeba3a1 09:31:15,814 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_22#324928838] 09:31:15,815 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_22#324928838] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:15,815 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_22#324928838, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0, version=12] 09:31:15,815 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:15,815 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_22#324928838)] 09:31:15,817 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 ReadData operation succeeded 09:31:15,817 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 closeTransaction called 09:31:15,817 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-10_22#324928838)] 09:31:16,319 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1659291375] 09:31:16,319 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 09:31:16,319 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] 09:31:16,319 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] 09:31:16,320 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 09:31:16,320 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:16,320 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:16,320 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] found - trying create transaction 09:31:16,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession)] 09:31:16,320 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Queuing TransactionOperation 09:31:16,322 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Creating transaction : member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 09:31:16,322 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} 09:31:16,322 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@1eeba3a1 09:31:16,322 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_23#1530662324] 09:31:16,325 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_23#1530662324, transactionId=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0, version=12] 09:31:16,325 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:16,325 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_23#1530662324)] 09:31:16,328 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 ReadData operation succeeded 09:31:16,328 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 closeTransaction called 09:31:16,328 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-cars-member-2:datastore-testSingleTransactionsWritesInQuickSuccession@0:1-11_23#1530662324)] 09:31:16,328 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:16,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:16,328 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 09:31:16,329 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 09:31:16,329 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:16,329 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 09:31:16,330 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 09:31:16,330 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 09:31:16,330 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:16,331 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:31:16,331 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:31:16,332 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:16,332 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:16,332 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 09:31:16,332 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1962668531] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:16,332 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1122432283] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1122432283] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20d44b3 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@20d44b3 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 09:31:16,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 8, leaderLastIndex: 7, leaderSnapShotIndex: 6 09:31:16,332 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=12] 09:31:16,332 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:16,332 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:16,334 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:16,334 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#-966976063] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:16,540 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:31:16,651 AM [cluster-test-shard-dispatcher-14] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Failed to transfer leadership in 318.8 ms 09:31:16,651 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In unpauseLeader 09:31:16,651 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader transfer failed - sending PoisonPill 09:31:16,651 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leadership transfer expired 09:31:16,652 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:31:16,652 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-727: Closing 09:31:16,652 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 09:31:16,653 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 09:31:16,653 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:31:16,653 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:31:16,653 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 09:31:16,653 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:31:16,654 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:16,654 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:16,662 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:16,663 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:16,663 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:16,663 AM [cluster-test-akka.actor.default-dispatcher-17] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:16,665 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:16,668 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection closed with error: Connection reset 09:31:16,673 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:16,673 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:17,670 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5235723619826004088] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:17,686 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:17,686 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:31:17.689] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:17,690 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3790287169391222277] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:17,701 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:17,701 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:17,703 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:17.704] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:17,718 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:31:17,724 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4246305009971286640] 09:31:17,724 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:17,727 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:17,727 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:17,727 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:17,727 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:17,741 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:17,741 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:17,741 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:17,742 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:17,747 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3760200477148559276] 09:31:17,748 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:17,750 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:17,750 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:17,761 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:17,761 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:17,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:17,782 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5340395534980020892] 09:31:17,782 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:17,786 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:17,786 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:17,790 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:17,790 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:17,795 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:17,796 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:17,797 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:17,801 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:17,805 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:17,814 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1801420010] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,815 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:17,817 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:17,817 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:31:17,817 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,817 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:17,817 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@40b40a98 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@31ddf87a 09:31:17,817 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:17,818 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,818 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58d480ed 09:31:17,818 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037477817, None)) 09:31:17,820 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037477816,0,0)) 09:31:17,820 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@45e80135 09:31:17,820 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@31ddf87a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@613fdb4b 09:31:17,820 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:31:17,820 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:17,822 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:17,824 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:17,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:17,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,825 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,825 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] 09:31:17,826 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:31:17,826 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:31:17,826 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,826 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#1334589628] 09:31:17,827 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1317832847] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,827 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:17,828 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:17,828 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:17,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:31:17,828 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:31:17,829 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:17,829 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:17,834 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1317832847 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,834 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:31:17,834 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:17,834 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#1334589628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,835 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,836 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:31:17,838 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:17,838 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,839 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:17,839 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,842 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:17,847 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:17,849 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:31:17,849 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:17,849 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3bf36f71 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb55abb 09:31:17,849 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2a8adbb5 09:31:17,850 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037477849, None)) 09:31:17,850 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037477848,0,0)) 09:31:17,850 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@46c30b11 09:31:17,850 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2eb55abb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a49312c 09:31:17,850 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:31:17,851 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:17,851 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,852 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,852 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,852 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-637459893] 09:31:17,852 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:31:17,852 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#-637459893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,853 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#859571317 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,853 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:31:17,853 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#859571317] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:17,854 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:31:17,854 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:17,854 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:31:17,854 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:17,854 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:31:17,857 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1320657514] 09:31:17,857 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#1320657514] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:17,901 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1801420010] 09:31:17,901 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 09:31:17,901 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] 09:31:18,106 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:18,106 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 09:31:18,106 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:31:18,106 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:18,107 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 09:31:18,107 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 09:31:18,357 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:31:18,357 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,357 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 09:31:18,390 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:18,657 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:31:18,657 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,657 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 09:31:18,737 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:18,738 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,738 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,739 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,746 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,808 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:18,896 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:31:18,896 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,896 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 09:31:18,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:18,900 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 09:31:18,901 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 09:31:18,901 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:31:18,902 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:18,903 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:31:18,904 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 09:31:18,904 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:18,907 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testRaftCallbackDuringLeadershipDrop commit rateLimit adjusted to 79.76650015107607 09:31:18,907 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,907 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1801420010] 09:31:18,907 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 09:31:18,907 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying 1 components for commit 09:31:18,907 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Readying transaction for shard cars 09:31:18,907 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 Queuing TransactionOperation 09:31:18,907 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 canCommit 09:31:18,907 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:18,907 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5b745e63] 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5b745e63] 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516)] for shard cars 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6274cab2 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 = [ 1 ] 09:31:18,908 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516)] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, disabling transaction tracking 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 validated 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: success 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 candidate ready 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: canCommit - messages: [] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 prepared 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:18,909 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 submitted to persistence 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:18,910 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}]]] 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:18,911 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,911 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:18,911 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=3083332} is already disabled 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 09:31:18,912 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}]]] 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:18,913 AM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 09:31:18,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:18,914 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%23-352458516$a] 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#341120219] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:31:18,914 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 successfully completed direct commit 09:31:18,914 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 committed successfully 09:31:18,914 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#341120219] 09:31:18,914 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:18,915 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:18,915 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:31:18,915 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 73.88 μs 09:31:18,915 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#341120219] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,915 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:18,916 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:18,918 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 09:31:18,921 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:18,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-2114387031] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:31:18,922 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-2114387031] 09:31:18,922 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:18,922 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:31:18,922 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 73.43 μs 09:31:18,922 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-2114387031] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,925 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:18,925 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1246921667] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,927 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:18,927 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:18,928 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,928 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:31:18,928 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:18,928 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@21fcca2c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@310bad90 09:31:18,928 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@56892194 09:31:18,930 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037478928, None)) 09:31:18,930 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037478927,0,0)) 09:31:18,931 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@310bad90 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6504414d 09:31:18,931 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@392f0dd7 09:31:18,931 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:31:18,931 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:18,931 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,931 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1246921667] 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1896254704] 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1896254704] 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,932 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1896254704] 09:31:18,932 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:31:18,933 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:31:18,933 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] 09:31:18,936 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#-1896254704] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,936 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1518980803] 09:31:18,937 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-1518980803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,938 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,938 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:18,938 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@6274cab2 09:31:18,938 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:18,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:31:18,940 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,940 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,942 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#401350441 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,942 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:31:18,942 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:31:18,942 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:31:18,942 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#401350441] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:18,943 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1801420010] 09:31:18,943 AM [pool-33-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying 1 components for commit 09:31:18,943 AM [pool-33-thread-1] [DEBUG] TransactionProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 Readying transaction for shard cars 09:31:18,943 AM [pool-33-thread-1] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 = [ 1 ] 09:31:18,943 AM [pool-33-thread-1] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516)] 09:31:18,943 AM [pool-33-thread-1] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 canCommit 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: handleReadyLocalTransaction for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying local modifications for Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 validated 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: canCommit for member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: success 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,943 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 candidate ready 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - candidate rootPath: / 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: canCommit - messages: [] 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 prepared 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Finishing commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 71 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]]] 09:31:18,944 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 submitted to persistence 09:31:18,949 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:19,44 AM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] 09:31:19,46 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-732: Closing 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 09:31:19,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,51 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 09:31:19,52 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 09:31:19,52 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:19,52 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:19,52 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 09:31:19,287 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:31:19,288 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 09:31:19,288 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:19,290 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 09:31:19,292 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:31:19,293 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:31:19,293 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=3] 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 09:31:19,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,294 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:31:19,294 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:19,294 AM [cluster-test-shard-dispatcher-25] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:19,295 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:31:19,301 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:31:19,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 09:31:19,301 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:31:19,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:31:19,301 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:19,302 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 09:31:19,302 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,302 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=3] 09:31:19,302 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=173} 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=71} 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63} 09:31:19,303 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} does not match any client, pre-disabling client. 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data Simple{size=173} 09:31:19,303 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:31:19,304 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:19,304 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1951262109] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 09:31:19,304 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1951262109] 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, size=71} 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:19,304 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0: commit - successfulFromPrevious: [] 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testRaftCallbackDuringLeadershipDrop_member-1-shard-cars-testRaftCallbackDuringLeadershipDrop%23-352458516$b] 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:31:19,304 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 131.6 μs 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}]] - lastApplied: 3, lastIndex: 3 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1951262109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-735: Closing 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:31:19,305 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 successfully completed direct commit 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 committed successfully 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 35.37 μs 09:31:19,305 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 09:31:19,306 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 09:31:19,306 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:31:19,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, size=63}] 09:31:19,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=173}] 09:31:19,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=71}] 09:31:19,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:31:19,307 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 09:31:19,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:19,307 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:31:19,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 09:31:19,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=71} 09:31:19,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:19,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:31:19,308 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:31:19,308 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:31:19,308 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:31:19,308 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:31:19,308 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 32.84 μs 09:31:19,308 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 53.42 μs 09:31:19,354 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:19,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,355 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:31:19,355 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:31:19,355 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,355 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:31:19,355 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:31:19,355 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:31:19,356 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:19,356 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,356 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 09:31:19,356 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,356 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:19,357 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1246921667] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1246921667] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,357 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,357 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:31:19,357 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:31:19,357 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,357 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:31:19,357 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:31:19,358 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:31:19,358 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:19,358 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,358 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:31:19,358 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,358 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 09:31:19,358 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,358 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 09:31:19,359 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1892856026] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,359 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#2082402665] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#2082402665] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42a32775 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@42a32775 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 09:31:19,359 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=12] 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:19,359 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:19,359 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:19,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:19,362 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,362 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#-352458516] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:19,657 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Failed to transfer leadership in 297.8 ms 09:31:19,657 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In unpauseLeader 09:31:19,657 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader transfer failed - sending PoisonPill 09:31:19,657 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leadership transfer expired 09:31:19,659 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:31:19,659 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-736: Closing 09:31:19,659 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:31:19,659 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:31:19,659 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:31:19,659 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,660 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:31:19,661 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:19,661 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:19,666 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:19,666 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:19,666 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:19,666 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:19,669 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:19,678 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:19,678 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:20,677 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4246305009971286640] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:20,692 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:20,692 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:20,694 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:20.697] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:20,700 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3760200477148559276] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:20,712 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:20,712 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:20,716 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:20.718] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:20,736 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:20,742 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2208276695170567650] 09:31:20,742 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:20,743 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:20,743 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:20,744 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:20,744 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:20,753 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 09:31:20,753 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_17#246721381)] 09:31:20,753 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 09:31:20,753 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testReadWriteTransactionWithMultipleShards@0:1_16#1039198389)] 09:31:20,761 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:20,761 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:20,761 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:20,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:20,772 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3805631647322366017] 09:31:20,772 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:20,772 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:20,772 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:20,783 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:20,783 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:20,788 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:20,819 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [520689123849687320] 09:31:20,820 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:20,822 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:20,822 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:20,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:20,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:20,833 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:20,834 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:20,836 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 09:31:20,843 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#766919343] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,843 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 09:31:20,854 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 09:31:20,854 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:20,854 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:20,854 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@26857c56 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1f104d4b 09:31:20,854 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1b2fda71 09:31:20,855 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1714037480855, None)) 09:31:20,856 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037480854,0,0)) 09:31:20,856 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1f104d4b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@521f168 09:31:20,856 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 09:31:20,856 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:20,857 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 09:31:20,857 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,858 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:20,858 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:20,858 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,858 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@e311d85 09:31:20,858 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:20,858 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,859 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,859 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,859 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-69779382] 09:31:20,859 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 09:31:20,860 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 09:31:20,861 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-69779382] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,862 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#2067917497 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,862 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#2067917497] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,862 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1508243558] 09:31:20,862 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#1508243558] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,863 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:20,863 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 09:31:20,864 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,866 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:20,866 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:20,866 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:20,866 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:31:20,866 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 09:31:20,866 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 09:31:20,866 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 09:31:20,866 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4401d898 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@48df4d9f 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@ddddae0 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1714037480868, None)) 09:31:20,868 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037480867,0,0)) 09:31:20,869 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@48df4d9f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35459934 09:31:20,869 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 09:31:20,870 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 09:31:20,870 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:20,871 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 09:31:20,872 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:20,872 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 09:31:20,872 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 09:31:20,872 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3fc746b7 09:31:20,873 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:20,873 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 09:31:20,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:20,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,873 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:20,873 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-866171579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,873 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 09:31:20,874 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1957482628] 09:31:20,874 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#1226289266 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 09:31:20,874 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 09:31:20,874 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#1226289266] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,874 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1957482628] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:20,874 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:31:20,875 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 09:31:20,875 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 09:31:20,875 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 09:31:20,875 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 09:31:20,919 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#766919343] 09:31:20,919 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 09:31:20,919 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-69779382] 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:21,141 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:31:21,141 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 09:31:21,142 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 09:31:21,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:31:21,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,451 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 09:31:21,473 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:21,721 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:31:21,721 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,721 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 09:31:21,762 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:21,762 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:21,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:21,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 09:31:21,762 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:21,762 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,763 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,763 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,770 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,836 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:21,961 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:31:21,961 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,961 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 09:31:21,961 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:21,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 09:31:21,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:21,963 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:21,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 09:31:21,964 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:21,964 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,964 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:31:21,965 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 09:31:21,965 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=12] 09:31:21,965 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:21,965 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:21,965 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 09:31:21,965 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 09:31:21,965 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 09:31:21,966 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 09:31:21,966 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:31:21,966 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=12] 09:31:21,966 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:21,966 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:21,966 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 09:31:21,966 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 09:31:21,967 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:31:21,967 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:21,975 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,975 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] 09:31:21,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:21,975 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:21,975 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 09:31:21,975 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=12] 09:31:21,975 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 09:31:21,975 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:21,975 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 09:31:21,975 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=12] 09:31:21,975 AM [main] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - waiting for ready futures with pending Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,975 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Queuing TransactionOperation 09:31:21,975 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 09:31:21,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - creating WRITE_ONLY transaction context 09:31:21,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:21,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 09:31:21,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 09:31:21,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, disabling transaction tracking 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 09:31:21,977 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@32acd3a1 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 of 1 operations, client version 12 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 ready 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 validated 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: success 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 candidate ready 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - candidate rootPath: / 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: canCommit - messages: [] 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 prepared 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}]]] 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 173 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,978 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 submitted to persistence 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 1 09:31:21,978 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:21,979 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 09:31:21,979 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62}] 09:31:21,979 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 09:31:21,979 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 09:31:21,979 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=2273579} is already disabled 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:31:21,979 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 09:31:21,980 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}]]] 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=173} 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 0 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, size=62} 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:21,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:31:21,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64}] 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 2, nextIndex: 3 09:31:21,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=173}] 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 09:31:21,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=0 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} has no callback 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0: commit - successfulFromPrevious: [] 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard$a] 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1436916654] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 09:31:21,982 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1436916654] 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:31:21,982 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:21,982 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 09:31:21,982 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 60.19 μs 09:31:21,982 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1436916654] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 2 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0}, size=64} 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 09:31:21,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data Simple{size=173} 09:31:21,984 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransactionChainProxy - Tx: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 - previous Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 readied - proceeding to FindPrimaryShard 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 modifications completed with CommitTransactionReply [version=12] 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 09:31:21,984 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1415186547] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 09:31:21,984 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1415186547] 09:31:21,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:31:21,985 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:31:21,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=2 09:31:21,985 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1415186547] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:21,985 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 59.70 μs 09:31:21,986 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 09:31:21,986 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 09:31:21,986 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@60c4f8a3 09:31:21,986 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_24#1308844452] 09:31:21,987 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_24#1308844452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:21,990 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_24#1308844452, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0, version=12] 09:31:21,990 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:21,990 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_24#1308844452)] 09:31:21,993 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 ReadData operation succeeded 09:31:21,993 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:21,993 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:21,993 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:21,993 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 09:31:21,993 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 09:31:21,993 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Queuing TransactionOperation 09:31:21,995 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:21,995 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 09:31:21,995 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@60c4f8a3 09:31:21,995 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadWriteTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057] 09:31:21,995 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:21,996 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, version=12] 09:31:21,996 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:21,996 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057)] 09:31:21,997 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 09:31:21,997 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:21,998 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:31:21,998 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead DataExists called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:31:21,998 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: false 09:31:21,998 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057)] 09:31:21,998 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.DataExists to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057)] 09:31:22,0 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with BatchedModificationsReply [numBatched=2] 09:31:22,0 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 DataExists operation succeeded 09:31:22,0 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:22,1 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057)] 09:31:22,3 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ReadData operation succeeded 09:31:22,3 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] 09:31:22,3 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 executeDelete called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] 09:31:22,3 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 09:31:22,3 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 successfully completed direct commit 09:31:22,3 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying 1 components for commit 09:31:22,3 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 Readying transaction for shard cars 09:31:22,3 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 directCommit called 09:31:22,3 AM [main] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 sending 2 batched modifications, ready: true 09:31:22,3 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-2_25#1572403057)] 09:31:22,3 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 canCommit 09:31:22,5 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadWriteTransaction - readyTransaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: handleForwardedReadyTransaction for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, client version 12 09:31:22,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,6 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 ready 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 validated 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: canCommit for member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: success 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Committing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 candidate ready 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - candidate rootPath: / 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: canCommit - messages: [] 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: preCommit - successfulFromPrevious: [] 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 prepared 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Finishing commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,7 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 09:31:22,8 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 submitted to persistence 09:31:22,9 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}]]] 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 1 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=229} 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 09:31:22,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:22,10 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=229}] 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0: commit - successfulFromPrevious: [] 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionChainWithSingleShard_member-1-shard-cars-testTransactionChainWithSingleShard_shard-cars-member-2%3Adatastore-testTransactionChainWithSingleShard%400%3A1-2_25%231572403057$i] 09:31:22,10 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 09:31:22,10 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 73.63 μs 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:22,10 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 09:31:22,11 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 successfully completed direct commit 09:31:22,11 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 modifications completed with CommitTransactionReply [version=12] 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:31:22,11 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=229} 09:31:22,11 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:22,11 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:22,11 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:31:22,11 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] found - trying create transaction 09:31:22,11 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard)] 09:31:22,12 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Queuing TransactionOperation 09:31:22,12 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:31:22,12 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 09:31:22,12 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 09:31:22,12 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 60.75 μs 09:31:22,13 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} 09:31:22,13 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Creating transaction : member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 09:31:22,14 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_26#732998702] 09:31:22,14 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_26#732998702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,15 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_26#732998702, transactionId=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0, version=12] 09:31:22,15 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:22,15 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_26#732998702)] 09:31:22,18 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 ReadData operation succeeded 09:31:22,18 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:22,19 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testTransactionChainWithSingleShard 09:31:22,19 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 09:31:22,19 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onShutDown 09:31:22,19 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 09:31:22,19 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 09:31:22,19 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 09:31:22,19 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 09:31:22,19 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 09:31:22,19 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:31:22,19 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:31:22,20 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: shutting down 09:31:22,20 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:31:22,20 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1022193790] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1022193790] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,20 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 09:31:22,20 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 09:31:22,20 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,20 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#2087083321] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f39aa13 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3f39aa13 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:22,20 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:22,22 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-69779382] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,22 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#-69779382] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:22,321 AM [cluster-test-shard-dispatcher-13] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionChainWithSingleShard: Failed to transfer leadership in 301.0 ms 09:31:22,321 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In unpauseLeader 09:31:22,321 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader transfer failed - sending PoisonPill 09:31:22,321 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leadership transfer expired 09:31:22,322 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 09:31:22,322 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-741: Closing 09:31:22,322 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 09:31:22,322 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 09:31:22,322 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 09:31:22,323 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:31:22,323 AM [cluster-test-akka.actor.default-dispatcher-16] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:31:22,324 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:22,324 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:22,331 AM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:22,331 AM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:22,331 AM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:22,331 AM [cluster-test-akka.actor.default-dispatcher-19] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:22,333 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:22,345 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:22,345 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:23,346 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2208276695170567650] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:23,365 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-3805631647322366017] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:23,366 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:23,366 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:23,368 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:23.370] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,384 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:23,384 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:23,387 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:23.390] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,421 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,429 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [7369618673054906329] 09:31:23,429 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:23,431 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:23,431 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:23,444 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,444 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,444 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:23,444 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:23,444 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:23,447 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,462 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5652400474144946147] 09:31:23,462 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:23,464 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,464 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:23,471 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,471 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,482 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,487 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4966730851297326555] 09:31:23,487 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:23,489 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,489 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:23,497 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,497 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,501 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:23,501 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:23,505 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:23,514 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:31:23,514 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:31:23,525 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:23,525 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:23,527 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:23.528] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config [WARN] [04/25/2024 09:31:23.539] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,559 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,567 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1664677042837474155] 09:31:23,567 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:23,567 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:23,567 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:23,574 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,574 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,576 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:23,576 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:23,576 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:23,579 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,597 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8493693240042333040] 09:31:23,597 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:23,599 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,599 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:23,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,605 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,618 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:23,628 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2231011486222890591] 09:31:23,628 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:23,630 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:23,630 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:23,640 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:23,642 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:23,642 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:23,644 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:23,645 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 09:31:23,652 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:23,658 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1129427383] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,658 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:23,659 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:23,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:31:23,662 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@180ed91b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@156bf3ff 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@50697f36 09:31:23,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:23,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1714037483662, None)) 09:31:23,662 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037483661,0,0)) 09:31:23,662 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@a3090c8 09:31:23,663 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@156bf3ff to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3b02594c 09:31:23,663 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 09:31:23,663 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:23,665 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:23,666 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 09:31:23,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:23,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,667 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,667 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764] 09:31:23,668 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 09:31:23,668 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 09:31:23,670 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,671 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:23,671 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1551653395 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,671 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1437215082] 09:31:23,671 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:23,674 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:23,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 09:31:23,674 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 09:31:23,674 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:23,674 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:31:23,674 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 09:31:23,674 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 09:31:23,674 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 09:31:23,678 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1551653395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,678 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#-1437215082] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,679 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:23,679 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,680 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:23,680 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@44053004 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ced1682 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@49ef997b 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1714037483681, None)) 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037483680,0,0)) 09:31:23,681 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ced1682 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5f0178be 09:31:23,681 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 09:31:23,682 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:23,684 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7c322877 09:31:23,684 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:23,684 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,685 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:23,686 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,686 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,686 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1523954272] 09:31:23,686 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 09:31:23,687 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 09:31:23,687 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1523954272] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,687 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#411882920] 09:31:23,688 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1613143840] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,688 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#1613143840 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,688 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#411882920] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:23,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 09:31:23,695 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,695 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 09:31:23,695 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 09:31:23,695 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 09:31:23,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1129427383] 09:31:23,732 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:31:23,732 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764] 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:31:23,963 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 09:31:23,963 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 09:31:24,217 AM [cluster-test-shard-dispatcher-25] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:24,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:31:24,253 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,253 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 09:31:24,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:31:24,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,482 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,603 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,603 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,603 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,603 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,614 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,655 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:24,792 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:31:24,792 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,792 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 09:31:24,792 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:24,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:24,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:24,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 09:31:24,800 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:24,800 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 09:31:24,801 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=12] 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 09:31:24,801 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 09:31:24,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 09:31:24,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=12] 09:31:24,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:24,802 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:24,802 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 09:31:24,802 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 09:31:24,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 09:31:24,802 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:24,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] 09:31:24,889 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:31:24,889 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1523954272] 09:31:24,889 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:24,889 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] 09:31:24,889 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:31:24,889 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1523954272] 09:31:24,993 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 09:31:25,0 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,0 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,0 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 09:31:25,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 09:31:25,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:31:25,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-79-0 of 3 operations, client version 12 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-79-0 ready 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 validated 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-79-0: success 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 candidate ready 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-79-0: canCommit - candidate rootPath: / 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-79-0: canCommit - messages: [] 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-79-0: preCommit - successfulFromPrevious: [] 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 prepared 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-79-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 submitted to persistence 09:31:25,16 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:25,16 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:25,16 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3915557} is already disabled 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:31:25,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:31:25,18 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:31:25,19 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:31:25,19 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-79-0: commit - successfulFromPrevious: [] 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 changing state from COMMIT_PENDING to COMMITTED 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-79-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-648#1263002537] 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#146778132] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 09:31:25,19 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#146778132] 09:31:25,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:31:25,19 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:31:25,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:25,20 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#146778132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:31:25,20 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 77.16 μs 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:31:25,21 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 09:31:25,21 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1129427383] 09:31:25,21 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:25,21 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 Queuing TransactionOperation 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-79-0 09:31:25,21 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@16d1c40b] 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@16d1c40b] 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] for shard cars 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1682517400] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a4b63c4 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1682517400] 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:31:25,21 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:31:25,21 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:31:25,22 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 36.99 μs 09:31:25,22 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1682517400] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,22 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,23 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 09:31:25,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-80-0 of 1 operations, client version 12 09:31:25,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-80-0 ready 09:31:25,25 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0 canCommit 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] resolved 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: successfully resolved all cohort actors 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0 finishCanCommit 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-80-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] 09:31:25,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 validated 09:31:25,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:25,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-80-0: success 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:31:25,25 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: canCommit returning result: true 09:31:25,25 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0 commit 09:31:25,25 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0 finish commit 09:31:25,25 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-80-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] 09:31:25,25 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 candidate ready 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-80-0: canCommit - candidate rootPath: / 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-80-0: canCommit - messages: [] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-80-0: preCommit - successfulFromPrevious: [] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 prepared 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-80-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:31:25,26 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 submitted to persistence 09:31:25,27 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 09:31:25,34 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:25,37 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-80-0 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-80-0: commit - successfulFromPrevious: [] 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 changing state from COMMIT_PENDING to COMMITTED 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-80-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23925693764$a] 09:31:25,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 09:31:25,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:25,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 09:31:25,43 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-80-0: commit succeeded 09:31:25,43 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 09:31:25,43 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:31:25,43 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:25,43 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@4a4b63c4 09:31:25,43 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0: Local transaction context created successfully, using DirectTransactionWrapper 09:31:25,43 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 70.43 μs 09:31:25,43 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:31:25,44 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,44 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 09:31:25,44 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 09:31:25,48 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,48 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 09:31:25,48 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:31:25,48 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:31:25,48 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:25,48 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:31:25,48 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6667e47 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6667e47 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 09:31:25,49 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:25,50 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=12] 09:31:25,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:25,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:25,50 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,50 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 09:31:25,50 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,50 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:31:25,51 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#559571485] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#559571485] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,51 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-671253666] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,52 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#925693764] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,342 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Failed to transfer leadership in 293.0 ms 09:31:25,342 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In unpauseLeader 09:31:25,342 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader transfer failed - sending PoisonPill 09:31:25,342 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leadership transfer expired 09:31:25,343 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:31:25,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-746: Closing 09:31:25,343 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:31:25,343 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:31:25,344 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:31:25,344 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:31:25,344 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1129427383] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-1129427383] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:25,344 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 09:31:25,344 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:31:25,345 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:25,345 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:25,350 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:25,350 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:25,350 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:25,350 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:25,352 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:25,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:25,363 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:26,365 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1664677042837474155] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:26,379 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:26,379 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:26,380 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:26.382] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:26,385 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8493693240042333040] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:26,389 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:26,389 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:26,391 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:26.392] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:26,401 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:26,412 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7393661097259534812] 09:31:26,412 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:26,413 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:26,413 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:26,420 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:26,420 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:26,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:26,425 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:26,426 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:26,432 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:26,440 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1371296634012028179] 09:31:26,441 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:26,442 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:26,442 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:26,448 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:26,448 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:26,460 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:26,468 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2397589402117376101] 09:31:26,468 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:26,469 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:26,469 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:26,469 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:26,469 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:26,477 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:26,482 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:26,484 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 09:31:26,485 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 closeTransaction called 09:31:26,485 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:1-1_24#1308844452)] 09:31:26,485 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 closeTransaction called 09:31:26,485 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-cars-member-2:datastore-testTransactionChainWithSingleShard@0:0_26#732998702)] 09:31:26,490 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-918964573] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,490 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:31:26,493 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 09:31:26,493 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:31:26,493 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6516dfbc to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ab4dc8e 09:31:26,493 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@a562212 09:31:26,493 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1714037486493, None)) 09:31:26,494 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037486492,0,0)) 09:31:26,494 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ab4dc8e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3af2f40 09:31:26,494 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@44fb5630 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:26,494 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,494 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:26,496 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:26,497 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 09:31:26,500 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,500 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] 09:31:26,500 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,500 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 09:31:26,501 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1729920153 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,501 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:31:26,501 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#1729920153] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,501 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#422723799] 09:31:26,501 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#422723799] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,501 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 09:31:26,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:26,503 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:26,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:26,503 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:31:26,503 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 09:31:26,503 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 09:31:26,503 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 09:31:26,503 AM [cluster-test-akka.actor.default-dispatcher-16] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 09:31:26,504 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:26,505 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 09:31:26,506 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,508 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:26,509 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 09:31:26,509 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:31:26,509 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@46062ecf to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@8fa32ec 09:31:26,509 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@cc5b72b 09:31:26,512 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1714037486511, None)) 09:31:26,512 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037486510,0,0)) 09:31:26,512 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@8fa32ec to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1f082647 09:31:26,513 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2fe4536b 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:26,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,515 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,515 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,515 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1023190630] 09:31:26,515 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 09:31:26,516 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 09:31:26,516 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-1023190630] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1118174473] 09:31:26,516 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-85275760 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,516 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-85275760] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,516 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1118174473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:26,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:26,518 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:26,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:26,518 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:31:26,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 09:31:26,518 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 09:31:26,518 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,521 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 09:31:26,521 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 09:31:26,521 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 09:31:26,522 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:26,525 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:26,563 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-918964573] 09:31:26,563 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 09:31:26,563 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:31:26,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:31:26,799 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 09:31:26,800 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 09:31:27,98 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:31:27,98 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,98 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 09:31:27,127 AM [cluster-test-shard-dispatcher-30] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:27,389 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:31:27,389 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,389 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 09:31:27,440 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:27,440 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,441 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,441 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,447 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,485 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:27,638 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:31:27,639 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,639 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 09:31:27,639 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,640 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,641 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 09:31:27,642 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 09:31:27,642 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=12] 09:31:27,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:27,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:27,643 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 09:31:27,643 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 09:31:27,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 09:31:27,643 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,643 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=12] 09:31:27,643 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:27,643 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:27,643 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 09:31:27,643 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 09:31:27,644 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:31:27,644 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:27,671 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 199745214, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.datastore.TransactionChainProxy@788ae7b8} 09:31:27,671 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testChainedTransactionFailureWithSingleShard commit rateLimit adjusted to 76.73491585673622 09:31:27,671 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 09:31:27,671 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] 09:31:27,671 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:27,671 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:27,671 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=12] 09:31:27,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=12] 09:31:27,672 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying 1 components for commit 09:31:27,672 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Readying transaction for shard cars 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] for shard cars 09:31:27,672 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Queuing TransactionOperation 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] found - creating WRITE_ONLY transaction context 09:31:27,672 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 executeMerge called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:27,672 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 canCommit 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 directCommit called 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 sending 1 batched modifications, ready: true 09:31:27,672 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard)] 09:31:27,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, disabling transaction tracking 09:31:27,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 09:31:27,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 09:31:27,673 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@58cbdab9 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying 1 batched modifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 09:31:27,674 AM [cluster-test-shard-dispatcher-26] [ERROR] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Error handling BatchedModifications for Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:31:27,675 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 09:31:27,675 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}] 09:31:27,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}]]] 09:31:27,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Number of entries to be appended = 1 09:31:27,678 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Log size is now 1 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71}] 09:31:27,679 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done true 09:31:27,679 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 modifications failed org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:31:27,679 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:31:27,680 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@7eac9cee failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema.]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1124) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.datastore.SingleCommitCohortProxy$1.onComplete(SingleCommitCohortProxy.java:60) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.modification.MergeModification.apply(MergeModification.java:48) at org.opendaylight.controller.cluster.datastore.CohortEntry.applyModifications(CohortEntry.java:102) at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleBatchedModifications(ShardCommitCoordinator.java:147) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModificationsLocal(Shard.java:751) at org.opendaylight.controller.cluster.datastore.Shard.handleBatchedModifications(Shard.java:774) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:376) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:31:27,681 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Setting last applied to 0 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Moving last applied index from -1 to 0 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, size=71} 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} does not match any client, pre-disabling client. 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=7930066} is already disabled 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 09:31:27,682 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persisting ApplyJournalEntries with index=0 09:31:27,682 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@7eac9cee is no longer running 09:31:27,682 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:31:27,683 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,683 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 09:31:27,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 09:31:27,683 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 09:31:27,683 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 09:31:27,683 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 09:31:27,683 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:31:27,684 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:31:27,684 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:31:27,684 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 09:31:27,684 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:31:27,684 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1249639722] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1249639722] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,684 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 09:31:27,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 09:31:27,684 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 09:31:27,684 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1515747048] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@212c4e09 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@212c4e09 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:27,685 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,686 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,879 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,879 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0}, size=73}]]] 09:31:27,881 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#276591320] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Failed to transfer leadership in 314.2 ms 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In unpauseLeader 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader transfer failed - sending PoisonPill 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leadership transfer expired 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-751: Closing 09:31:27,999 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 09:31:28,0 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 09:31:28,0 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:31:28,0 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:31:28,0 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 09:31:28,0 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:31:28,0 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-918964573] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#-918964573] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:28,1 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:28,1 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:28,6 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:28,6 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:28,6 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:28,6 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:28,8 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:28,21 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:28,21 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:29,15 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7393661097259534812] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:29,35 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1371296634012028179] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:29,40 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:29,41 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:29,43 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:29.045] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:29,58 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:29,58 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:29,61 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:29.063] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:29,81 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:29,87 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1250807561346005990] 09:31:29,87 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:29,89 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:29,89 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:29,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:29,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:29,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:29,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:29,99 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:29,106 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:29,113 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6888430207672648973] 09:31:29,113 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:29,121 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:29,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:29,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:29,121 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:29,137 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:29,157 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5221426091824094689] 09:31:29,157 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:29,158 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:29,158 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:29,166 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:29,166 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:29,171 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:29,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:29,173 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 09:31:29,179 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:31:29,179 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#315329292] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,182 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 09:31:29,182 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,182 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:29,182 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4a73d9d1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64965f25 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@76e24b56 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1714037489183, None)) 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037489182,0,0)) 09:31:29,183 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@64965f25 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@274073eb 09:31:29,183 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@18ca2426 09:31:29,183 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 09:31:29,183 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:29,185 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:29,186 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 09:31:29,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:29,186 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,187 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,187 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1828845524] 09:31:29,188 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 09:31:29,188 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 09:31:29,188 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1828845524] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,188 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-825743770 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,188 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#-825743770] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,188 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1916700408] 09:31:29,189 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,189 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#-1916700408] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:31:29,190 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 09:31:29,190 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 09:31:29,190 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 09:31:29,191 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 09:31:29,196 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:31:29,202 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 09:31:29,202 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,202 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 09:31:29,202 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:29,202 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@21a3efbb to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ae33515 09:31:29,203 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7edcae91 09:31:29,203 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1714037489203, None)) 09:31:29,204 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037489202,0,0)) 09:31:29,204 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7ae33515 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@140574e1 09:31:29,204 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 09:31:29,204 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3a1e218d 09:31:29,205 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:29,205 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,205 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 09:31:29,205 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#1965464150] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,205 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#2121145235 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,205 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#2121145235] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,205 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 09:31:29,205 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#1350090330] 09:31:29,205 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#1350090330] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:29,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:29,207 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:29,207 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 09:31:29,207 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 09:31:29,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:31:29,207 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:31:29,207 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 09:31:29,207 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 09:31:29,207 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 09:31:29,209 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:29,209 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:29,211 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:29,211 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:29,211 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,211 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,211 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,212 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:29,254 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#315329292] 09:31:29,255 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 09:31:29,255 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1828845524] 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:31:29,459 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 09:31:29,459 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 09:31:29,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:31:29,769 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:29,769 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 09:31:30,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:31:30,9 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,9 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 09:31:30,110 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:30,110 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,110 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,110 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,132 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,180 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:30,249 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:31:30,249 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,249 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 09:31:30,249 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,250 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:30,250 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 09:31:30,250 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:30,250 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:30,250 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 09:31:30,251 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:30,251 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,251 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 09:31:30,255 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=12] 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 09:31:30,255 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=12] 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:30,256 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 09:31:30,256 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 09:31:30,256 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:30,261 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 09:31:30,262 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] 09:31:30,262 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Queuing TransactionOperation 09:31:30,262 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:30,262 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=12] 09:31:30,262 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=12] 09:31:30,262 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars 09:31:30,262 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,262 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, disabling transaction tracking 09:31:30,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 09:31:30,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 09:31:30,263 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:30,264 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:30,264 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,264 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 09:31:30,264 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Creating transaction : member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 09:31:30,264 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_27#1856747109] 09:31:30,264 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_27#1856747109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:30,264 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 09:31:30,264 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}]]] 09:31:30,264 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Number of entries to be appended = 1 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Log size is now 1 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:30,265 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_27#1856747109, transactionId=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0, version=12] 09:31:30,265 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:30,265 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_27#1856747109)] 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70}] 09:31:30,265 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done true 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: 0, nextIndex: 1 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Setting last applied to 0 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=5743178} is already disabled 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:30,269 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 09:31:30,270 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 ReadData operation succeeded 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Commit index set to 0 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Setting last applied to 0 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Moving last applied index from -1 to 0 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, size=70} 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} does not match any client, pre-disabling client. 09:31:30,270 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Persisting ApplyJournalEntries with index=0 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,270 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-756: Closing 09:31:30,271 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 09:31:30,271 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:31:30,271 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:30,271 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:30,279 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:30,279 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:30,279 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:30,279 AM [cluster-test-akka.actor.default-dispatcher-18] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:30,281 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:30,289 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 09:31:30,290 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithShardLeaderNotResponding commit rateLimit adjusted to 76.73491585673622 09:31:30,290 AM [main] [DEBUG] TransactionProxy - New READ_WRITE Tx - member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 09:31:30,290 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:30,290 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:30,290 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,290 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,290 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,290 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 09:31:30,290 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:30,290 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying 1 components for commit 09:31:30,290 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Readying transaction for shard cars 09:31:30,290 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Queuing TransactionOperation 09:31:30,290 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 canCommit 09:31:30,359 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:30,389 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:30,389 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,389 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,459 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:30,489 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:30,489 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,489 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,559 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:30,589 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:30,589 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,589 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,659 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0: create tx on shard cars failed with exception "akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply." - scheduling retry in 10 ms 09:31:30,689 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Retrying findPrimaryShardAsync for shard cars 09:31:30,689 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] found - trying create transaction 09:31:30,689 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)] 09:31:30,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 Creating NoOpTransaction because of error akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:31:30,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:30,759 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] NoOpTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 directCommit called, failure org.opendaylight.controller.cluster.datastore.exceptions.ShardLeaderNotRespondingException: Could not create a READ_WRITE transaction on shard cars. The shard leader isn't responding. at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.createTransactionContext(RemoteTransactionContextSupport.java:213) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport.onCreateTransactionComplete(RemoteTransactionContextSupport.java:195) at org.opendaylight.controller.cluster.datastore.RemoteTransactionContextSupport$1.onComplete(RemoteTransactionContextSupport.java:136) at akka.dispatch.OnComplete.internal(Future.scala:299) at akka.dispatch.OnComplete.internal(Future.scala:297) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:224) at akka.dispatch.japi$CallbackBridge.apply(Future.scala:221) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484) at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63) at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94) at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding)]] after [50 ms]. Message of type [org.opendaylight.controller.cluster.datastore.messages.CreateTransaction]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:31:30,760 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:30,761 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,761 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithShardLeaderNotResponding - waiting on 1 shards 09:31:30,761 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onShutDown 09:31:30,761 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:31:30,761 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 09:31:30,761 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 09:31:30,762 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All shards shutdown - sending PoisonPill to self 09:31:30,762 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:31:30,762 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:31:30,762 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:31:30,762 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: shutting down 09:31:30,762 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:31:30,762 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:31:30,763 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:31:30,762 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:30,763 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#293429116] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:30,763 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1634923299] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1634923299] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:30,763 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:30,763 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:31,289 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1250807561346005990] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:31,779 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6888430207672648973] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:31,781 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:31,781 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:31:31.784] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:31,800 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:31,800 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:31,817 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:31.818] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:31,828 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:31:31,832 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2645342060054616785] 09:31:31,832 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:31,834 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:31,850 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:31,850 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:31,850 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:31,859 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:31,867 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4989924945093956540] 09:31:31,868 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:31,869 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:31,869 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:31,870 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:31,870 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:31,893 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:31,904 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5959236385060253043] 09:31:31,904 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:31,906 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:31,906 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:31,917 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:31,918 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:31,919 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:31,925 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#191423787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,926 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:31,928 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:31,931 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:31,931 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@307faf7d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@45f65830 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5c8c0d92 09:31:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:31,936 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:31,936 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037491936, None)) 09:31:31,936 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037491935,0,0)) 09:31:31,937 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@45f65830 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d845bd6 09:31:31,937 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 09:31:31,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:31,938 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:31,938 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:31,940 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:31,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2664ddbc 09:31:31,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:31,941 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,943 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,943 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,943 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#574263387] 09:31:31,944 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:31:31,944 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#574263387] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,944 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1087839154] 09:31:31,944 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-684074024] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,950 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:31:31,950 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:31,951 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:31,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:31,951 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:31,951 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:31:31,951 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:31:31,952 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-684074024 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,952 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:31:31,952 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:31,952 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:31:31,953 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1087839154] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,954 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:31,956 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@25af7090 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30783720 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@757abf4b 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037491959, None)) 09:31:31,959 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037491958,0,0)) 09:31:31,960 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@30783720 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2f801c00 09:31:31,960 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@63d949b9 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:31,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,961 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,961 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,961 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] 09:31:31,961 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:31:31,961 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,961 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1045005989] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,961 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-1045005989 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,961 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#544501062] 09:31:31,961 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:31:31,961 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#544501062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:31:31,962 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:31:31,963 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:31:31,963 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:31,966 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:31:31,979 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:31,981 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:32,10 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#191423787] 09:31:32,10 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:31:32,10 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#574263387] 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:32,245 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:32,246 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:31:32,246 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:31:32,504 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:32,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:32,564 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,564 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 09:31:32,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:32,795 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,795 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:32,846 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:32,846 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,846 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,846 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:32,925 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:33,105 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:33,105 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,105 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 09:31:33,105 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:33,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:31:33,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:33,107 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:33,107 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 09:31:33,108 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:33,108 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,108 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 09:31:33,109 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:33,109 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 09:31:33,110 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:31:33,110 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:33,110 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,110 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:31:33,110 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:33,110 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:33,110 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:31:33,110 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 09:31:33,111 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 09:31:33,111 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:33,125 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testSingleShardTransactionsWithLeaderChanges commit rateLimit adjusted to 76.73491585673622 09:31:33,125 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,125 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] 09:31:33,125 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 09:31:33,125 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 09:31:33,125 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying 1 components for commit 09:31:33,125 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Readying transaction for shard cars 09:31:33,125 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Queuing TransactionOperation 09:31:33,125 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 canCommit 09:31:33,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:33,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=12] 09:31:33,127 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=12] 09:31:33,127 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] for shard cars 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] found - creating WRITE_ONLY transaction context 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 directCommit called 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 sending 2 batched modifications, ready: true 09:31:33,128 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges)] 09:31:33,129 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, disabling transaction tracking 09:31:33,129 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 09:31:33,129 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 09:31:33,129 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying 2 batched modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Readying Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 of 2 operations, client version 12 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 ready 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 validated 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: success 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 candidate ready 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: canCommit - messages: [] 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 prepared 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,130 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,131 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 submitted to persistence 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}]]] 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 09:31:33,131 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:33,132 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71}] 09:31:33,132 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=2892340} is already disabled 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 09:31:33,132 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=181} 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:33,133 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} does not match any client, pre-disabling client. 09:31:33,134 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:33,134 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 1 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:33,134 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges$a] 09:31:33,135 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#550687320] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 09:31:33,135 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#550687320] 09:31:33,135 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 09:31:33,135 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:31:33,135 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:33,135 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 09:31:33,135 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 92.75 μs 09:31:33,135 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#550687320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,136 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 successfully completed direct commit 09:31:33,136 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 modifications completed with CommitTransactionReply [version=12] 09:31:33,136 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 1 09:31:33,136 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data Simple{size=181} 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1907272747] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 4 09:31:33,137 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1907272747] 09:31:33,137 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=1 09:31:33,137 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:31:33,137 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1907272747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,137 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 92.82 μs 09:31:33,138 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 09:31:33,138 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:33,138 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:33,139 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-761: Closing 09:31:33,139 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:31:33,139 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:33,139 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:33,139 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:33,145 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:33,145 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:33,145 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:33,145 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:33,147 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:33,157 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:31:33,207 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] 09:31:33,207 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:31:33,207 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] 09:31:33,207 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,257 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:33,264 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:33,265 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:31:33,265 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 09:31:33,265 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:33,265 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:33,265 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:31:33,313 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:33,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [2614821694318534745] 09:31:33,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:33,322 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:33,322 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:33,322 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:33,322 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:33,331 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:33,332 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:33,333 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:33,339 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:33,340 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1496233170] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:33,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 09:31:33,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@38a28e64 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@18c4422 09:31:33,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@448eda77 09:31:33,342 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037493342, None)) 09:31:33,343 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037493341,0,0)) 09:31:33,343 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@18c4422 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@19ae2243 09:31:33,343 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 09:31:33,345 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:33,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@783a8d99 09:31:33,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:33,345 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,346 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:31:33,346 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-2113510078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,346 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1097750438 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,346 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-775118156] 09:31:33,346 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#1097750438] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,346 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:31:33,346 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#-775118156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 71 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 181 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: ignoring unhandled payload DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, size=71} 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 1 to 1 09:31:33,348 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 09:31:33,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@690c2abd}, purged=MutableUnsignedLongSet{size=0}} 09:31:33,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:31:33,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 09:31:33,349 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:33,350 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 611.0 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 09:31:33,351 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:31:33,351 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:31:33,351 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:33,351 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:33,351 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:31:33,351 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:31:33,351 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:31:33,393 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] 09:31:33,394 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:31:33,394 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] 09:31:33,484 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:33,484 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,484 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 09:31:33,484 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,664 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:33,664 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,664 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 09:31:33,664 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,834 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:33,834 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,834 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 09:31:33,834 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:33,871 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:33,871 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:31:33,872 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:31:34,24 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:34,24 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,24 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 09:31:34,24 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,155 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2645342060054616785] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:34,156 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2645342060054616785] but current is [2614821694318534745]. Reason: ActorSystem terminated 09:31:34,164 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:31:34,164 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,164 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 09:31:34,164 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,231 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 5 09:31:34,231 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:31:34,232 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:31:34,232 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] LocalActorRefProvider$$anon$2 - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [4989924945093956540]. This system with UID [2614821694318534745] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,233 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 6 - updating term 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:34,233 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-1705191802]] intended for system uid [-2645342060054616785], self uid is [2614821694318534745] 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,233 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=true] 09:31:34,234 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2557/system/clusterReceptionist/replicator#-904267816]] intended for system uid [-2645342060054616785], self uid is [2614821694318534745] 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 7 - updating term 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=true] 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 8 - updating term 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,235 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=true] 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 9 - updating term 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,236 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=true] 09:31:34,239 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 10 09:31:34,239 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 09:31:34,239 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:31:34,239 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:31:34,239 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:34,239 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 10 09:31:34,240 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:31:34,240 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 09:31:34,240 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 09:31:34,240 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:31:34,240 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:34,240 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:34,240 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 09:31:34,240 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:31:34,240 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = null 09:31:34,242 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:34,242 AM [cluster-test-shard-dispatcher-16] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:34,243 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 1, nextIndex: 2 09:31:34,243 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:34,243 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 0 for term 10 09:31:34,243 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 09:31:34,249 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,249 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] 09:31:34,249 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 09:31:34,249 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying 1 components for commit 09:31:34,249 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Readying transaction for shard cars 09:31:34,249 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 Queuing TransactionOperation 09:31:34,249 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 canCommit 09:31:34,249 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:34,249 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2a5aada9] 09:31:34,249 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@2a5aada9] 09:31:34,249 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486)] for shard cars 09:31:34,249 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:31:34,249 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@62174b25 09:31:34,250 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 = [ 1 ] 09:31:34,250 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486)] 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: handleReadyLocalTransaction for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying local modifications for Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 validated 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: canCommit for member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: success 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 candidate ready 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - candidate rootPath: / 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: canCommit - messages: [] 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: preCommit - successfulFromPrevious: [] 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 prepared 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Finishing commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,250 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 240 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 submitted to persistence 09:31:34,251 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, entries=[SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}]]] 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - New leader address: akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:34,252 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=240} 09:31:34,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 09:31:34,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 09:31:34,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=10, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=10, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:34,253 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=10, payload=Simple{size=240}] 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 2 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data Simple{size=240} 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0: commit - successfulFromPrevious: [] 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 changing state from COMMIT_PENDING to COMMITTED 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testSingleShardTransactionsWithLeaderChanges_member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges%23954153486$b] 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 10 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:34,254 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 09:31:34,254 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SingleCommitCohortProxy - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 successfully completed direct commit 09:31:34,254 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 committed successfully 09:31:34,254 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:31:34,255 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 62.08 μs 09:31:34,255 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 09:31:34,255 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:34,255 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@62174b25 09:31:34,255 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0: Local transaction context created successfully, using DirectTransactionWrapper 09:31:34,256 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 09:31:34,256 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,256 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 09:31:34,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 09:31:34,256 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:31:34,256 AM [cluster-test-shard-dispatcher-16] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:31:34,257 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 09:31:34,257 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:34,257 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,257 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:34,257 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 09:31:34,257 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,257 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,257 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 09:31:34,257 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 09:31:34,257 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Initiating leader transfer 09:31:34,257 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=12] 09:31:34,257 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1233a61e 09:31:34,257 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:34,257 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:34,257 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1233a61e 09:31:34,258 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Attempting to transfer leadership 09:31:34,258 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1496233170] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1496233170] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,258 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:34,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1496233170] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1496233170] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,258 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1941247782] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1941247782] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,259 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,259 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,343 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:31:34,463 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 09:31:34,464 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#954153486]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Failed to transfer leadership in 206.5 ms 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: In unpauseLeader 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader transfer failed - sending PoisonPill 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-764: Closing 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:31:34,464 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:31:34,465 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,465 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:31:34,465 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStore.close(DistributedDataStore.java:70) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:31:34,465 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,465 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,465 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 09:31:34,466 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:31:34,466 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1731430526] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1731430526] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,466 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-1326692857] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:34,466 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:34,466 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:34,471 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:34,471 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:34,471 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:34,472 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:34,474 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:34,482 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:34,482 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:34,965 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 09:31:34,967 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2645342060054616785] but current is [2614821694318534745]. Reason: Cluster member removed, previous status [Down] 09:31:35,294 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [-2645342060054616785] but current is [2614821694318534745]. Reason: Cluster member removed, previous status [Down] 09:31:35,484 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [2614821694318534745] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:35,495 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4989924945093956540] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:35,495 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:35,495 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:35,499 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:35.500] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:35,515 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:35,515 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:35,518 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:35.519] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:35,533 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:35,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4203785256666927621] 09:31:35,541 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:35,544 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:35,544 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:35,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:35,554 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:35,557 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:35,557 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:35,557 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:35,566 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:35,574 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-572555532438844190] 09:31:35,574 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:35,576 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:35,576 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:35,584 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:35,584 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:35,599 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:35,621 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-715293372327306495] 09:31:35,621 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:35,626 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:35,626 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:35,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:35,627 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:35,645 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 closeTransaction called 09:31:35,645 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-cars-member-2:datastore-testTransactionWithShardLeaderNotResponding@0:0_27#1856747109)] 09:31:35,649 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:35,650 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:35,652 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:31:35,659 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:35,662 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,665 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 09:31:35,665 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:35,665 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6ea68d81 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@794bff38 09:31:35,666 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3c497e75 09:31:35,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:31:35,666 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:35,666 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,666 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037495666, None)) 09:31:35,669 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037495665,0,0)) 09:31:35,669 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@350007f8 09:31:35,669 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@794bff38 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@218d5714 09:31:35,669 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 09:31:35,669 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:35,671 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:35,673 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:31:35,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:35,673 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,674 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,674 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:35,674 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:31:35,674 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,675 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#462841185 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,675 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#462841185] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,675 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#816874381] 09:31:35,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#816874381] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,681 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:31:35,683 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1435481714] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:31:35,683 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:31:35,683 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:31:35,683 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 09:31:35,683 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:31:35,683 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:31:35,689 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:31:35,689 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5552c53f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@48203ae6 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@195975da 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037495690, None)) 09:31:35,690 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037495689,0,0)) 09:31:35,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5c2a5900 09:31:35,691 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@48203ae6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3c93356e 09:31:35,691 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 09:31:35,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:35,691 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:35,692 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:31:35,692 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1286010908] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,692 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:31:35,692 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-2032355293] 09:31:35,692 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-571932606] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:31:35,693 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 09:31:35,694 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:35,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-571932606 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:35,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:31:35,695 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:31:35,695 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-2032355293] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:35,695 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:31:35,700 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:35,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:35,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:35,700 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,704 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:35,706 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:35,741 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] 09:31:35,741 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 09:31:35,742 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:35,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:35,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 09:31:35,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:31:35,921 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:35,921 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 09:31:35,922 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:35,922 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:31:35,922 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:35,922 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:31:35,922 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 09:31:35,922 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 09:31:36,228 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:36,241 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:31:36,241 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,241 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 09:31:36,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:31:36,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,562 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 09:31:36,562 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:36,562 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,563 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,569 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,650 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:36,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:31:36,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,831 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 09:31:36,831 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:36,834 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 09:31:36,835 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 09:31:36,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:31:36,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:36,835 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:36,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 09:31:36,835 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:31:36,835 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 09:31:36,835 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,836 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:31:36,836 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:36,836 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:36,836 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:31:36,836 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 09:31:36,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 09:31:36,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:36,849 AM [main] [DEBUG] TransactionRateLimiter - On unused Tx - data Store testTransactionWithIsolatedLeader commit rateLimit adjusted to 76.73491585673622 09:31:36,849 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] 09:31:36,849 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Queuing TransactionOperation 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:36,849 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 09:31:36,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,849 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Queuing TransactionOperation 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:36,849 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,849 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] 09:31:36,849 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Queuing TransactionOperation 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:36,849 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] for shard cars 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Shard cars resolved to local data tree - adding local factory 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3e7e53c2 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] for shard cars 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3e7e53c2 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@15bc2343] 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: Found primary ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] for shard cars 09:31:36,850 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractTransactionContextFactory - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 - Creating local component for shard cars using factory org.opendaylight.controller.cluster.datastore.LocalTransactionFactoryImpl@3e7e53c2 09:31:36,860 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:36,860 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,860 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:31:36,860 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:31:36,861 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,861 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:31:36,861 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:31:36,861 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:31:36,861 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:36,861 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:36,861 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying 1 components for commit 09:31:36,861 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 Readying transaction for shard cars 09:31:36,861 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 = [ 1 ] 09:31:36,862 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, disabling transaction tracking 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,862 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}]]] 09:31:36,862 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1435481714] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1435481714] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 validated 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: success 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,862 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:31:36,862 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 candidate ready 09:31:36,862 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: canCommit - messages: [] 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 prepared 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:36,863 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#963288112] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#963288112] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 submitted to persistence 09:31:36,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:36,863 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:31:36,863 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1697514094] 09:31:36,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 09:31:36,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1435481714] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#-1435481714] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:36,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 09:31:36,864 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:36,864 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 09:31:36,864 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:36,865 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:36,915 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:36,951 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:36,965 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:37,16 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:37,66 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:37,72 AM [cluster-test-shard-dispatcher-12] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 09:31:37,73 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:37,73 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:31:37,73 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:37,74 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 09:31:37,74 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-769: Closing 09:31:37,74 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:37,74 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 09:31:37,74 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 09:31:37,75 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:37,117 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] 09:31:37,117 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying 1 components for commit 09:31:37,117 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 Readying transaction for shard cars 09:31:37,117 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 = [ 1 ] 09:31:37,117 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] 09:31:37,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 09:31:37,118 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@20002235 to retry 09:31:37,118 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 canCommit 09:31:37,531 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Timer expired for message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@20002235 09:31:37,531 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] LocalThreePhaseCommitCohort - Failed to prepare transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 on backend org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Could not process ready local transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0. Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport$MessageInfo.timedOut(ShardTransactionMessageRetrySupport.java:102) at org.opendaylight.controller.cluster.datastore.ShardTransactionMessageRetrySupport.onTimerMessage(ShardTransactionMessageRetrySupport.java:72) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:413) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:31:37,532 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 09:31:37,532 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying 1 components for commit 09:31:37,532 AM [main] [DEBUG] TransactionProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 Readying transaction for shard cars 09:31:37,532 AM [main] [DEBUG] TransactionContext - Total modifications on Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 = [ 1 ] 09:31:37,532 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,532 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869)] 09:31:37,532 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:31:37,532 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,532 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:37,532 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Adding message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@310eae7e to retry 09:31:37,534 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:37,536 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:31:37,538 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:37,538 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:37,539 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 09:31:37,539 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6f2cc6b2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7a2a82d7 09:31:37,539 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@124f95f3 09:31:37,540 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037497540, None)) 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:37,540 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,541 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037497539,0,0)) 09:31:37,541 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@29f788c1 09:31:37,541 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7a2a82d7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@7861245b 09:31:37,541 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:37,542 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 canCommit 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1254410478] 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1254410478] 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,542 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-1254410478] 09:31:37,542 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:31:37,543 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-346332913 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,543 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#-587378334] 09:31:37,543 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:37,544 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:31:37,544 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:31:37,544 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,544 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:31:37,544 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:31:37,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,551 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}]]] 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=170} 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 09:31:37,555 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:37,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 09:31:37,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60}] 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=170}] 09:31:37,556 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 1, nextIndex: 2 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 0 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=696762652} is already disabled 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Setting last applied to 1 09:31:37,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-833091869$a] 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1043119305] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 09:31:37,559 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 committed successfully 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:37,559 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1043119305] 09:31:37,559 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 09:31:37,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-770: Closing 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:37,560 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1043119305] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardTransactionMessageRetrySupport - member-1-shard-cars-testTransactionWithIsolatedLeader: Retrying message org.opendaylight.controller.cluster.datastore.messages.ReadyLocalTransaction@310eae7e 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 09:31:37,560 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 75.96 μs 09:31:37,560 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: handleReadyLocalTransaction for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying local modifications for Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 validated 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: canCommit for member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: success 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Committing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 candidate ready 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - candidate rootPath: / 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: canCommit - messages: [] 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: preCommit - successfulFromPrevious: [] 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 prepared 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Finishing commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 68 09:31:37,560 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 09:31:37,560 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:31:37,560 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 successfully completed direct commit 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:31:37,560 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,560 AM [main] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 canCommit 09:31:37,561 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 09:31:37,561 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 submitted to persistence 09:31:37,561 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 1 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, size=60} 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} does not match any client, pre-disabling client. 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 09:31:37,562 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data Simple{size=170} 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-2096750945] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 09:31:37,563 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#-2096750945] 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 0 for term 4 09:31:37,563 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:31:37,563 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 49.91 μs 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}]]] 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=68} 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 2 09:31:37,563 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:37,564 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=68}] 09:31:37,564 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=1 09:31:37,564 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 09:31:37,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,570 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=68} 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0: commit - successfulFromPrevious: [] 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 changing state from COMMIT_PENDING to COMMITTED 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testTransactionWithIsolatedLeader_member-1-shard-cars-testTransactionWithIsolatedLeader%23-833091869$b] 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:37,571 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 09:31:37,571 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SingleCommitCohortProxy - Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 successfully completed direct commit 09:31:37,571 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalThreePhaseCommitCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 committed successfully 09:31:37,571 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:31:37,571 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 17.29 μs 09:31:37,572 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 09:31:37,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,572 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:31:37,572 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:31:37,572 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:31:37,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:31:37,572 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:31:37,573 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:31:37,573 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:37,573 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:37,573 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:31:37,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:31:37,573 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:37,573 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:37,573 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:31:37,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:31:37,573 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 09:31:37,573 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=12] 09:31:37,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37a11419 09:31:37,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37a11419 09:31:37,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 09:31:37,574 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:37,574 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:37,574 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:37,581 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Failed to transfer leadership in 10.06 s 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In unpauseLeader 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer failed - sending PoisonPill 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leadership transfer expired 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:31:47,631 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-773: Closing 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:47,632 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.notifications.RoleChanged] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-833091869] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#462841185] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#462841185] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-774: Closing 09:31:47,632 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:31:47,633 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:31:47,633 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:31:47,633 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:47,633 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:31:47,633 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:31:47,634 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:47,634 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:47,638 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:47,638 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:47,638 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:47,638 AM [cluster-test-akka.actor.default-dispatcher-15] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:47,641 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:47,652 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:47,652 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:48,649 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4203785256666927621] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:48,670 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:48,670 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:48,671 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-572555532438844190] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:48,672 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:48.674] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:48,690 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:48,690 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:48,692 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:48.693] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:48,717 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:48,723 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8390208602871671502] 09:31:48,723 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:48,733 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:48,733 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:48,737 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:48,737 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:48,749 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:48,749 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:48,749 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:48,752 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:48,757 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2318459049985395869] 09:31:48,757 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:48,758 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:48,758 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:48,766 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:48,766 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:48,775 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:48,792 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1614253709848372383] 09:31:48,792 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:48,793 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:48,793 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:48,805 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:48,806 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:48,807 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 09:31:48,809 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:48,809 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:48,813 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-877532440] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,814 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,816 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@331b6295 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12c9f9c1 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3fd5b1c4 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1714037508824, None)) 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037508823,0,0)) 09:31:48,824 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3c4419b3 09:31:48,824 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12c9f9c1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1f826dbd 09:31:48,825 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 09:31:48,825 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:48,826 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:48,828 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 09:31:48,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:48,829 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,830 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:48,832 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:48,832 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,832 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,832 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074] 09:31:48,832 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493] 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 09:31:48,833 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 09:31:48,834 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1030609660 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-526490450] 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#1685404347 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1135211037] 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#-1030609660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-526490450] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#1685404347] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,835 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1135211037] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,836 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1365243954] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,836 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:48,836 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:48,836 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:31:48,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:48,836 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:48,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:48,836 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:31:48,836 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:48,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 09:31:48,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 09:31:48,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 09:31:48,837 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 09:31:48,839 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:31:48,839 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:48,839 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2562c53d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4915e151 09:31:48,840 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:48,840 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,840 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,841 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@c176617 09:31:48,842 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1714037508839, None)) 09:31:48,842 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037508838,0,0)) 09:31:48,842 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@37f95574 09:31:48,842 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4915e151 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3d510a76 09:31:48,842 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 09:31:48,843 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:48,843 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,843 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:48,843 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:31:48,843 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:31:48,845 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,845 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,845 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-1537114395] 09:31:48,845 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:48,845 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:48,845 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 09:31:48,845 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#-1537114395] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,845 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 09:31:48,846 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1679160096] 09:31:48,846 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-406739559 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,846 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 09:31:48,846 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-406739559] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,846 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#45681005 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:48,846 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1480794836] 09:31:48,847 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:48,848 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:48,848 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1067989923] 09:31:48,848 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 09:31:48,848 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 09:31:48,849 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1679160096] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,849 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#1067989923] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,849 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#45681005] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,849 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1480794836] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:48,849 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 09:31:48,849 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:48,849 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:48,849 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:31:48,850 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 09:31:48,850 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:48,850 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:48,850 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:31:48,850 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:31:48,850 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 09:31:48,850 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 09:31:48,850 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:48,850 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:31:48,850 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 09:31:48,893 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-877532440] 09:31:48,893 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 09:31:48,893 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074] 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:31:49,85 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:31:49,85 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:49,144 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:31:49,145 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:31:49,145 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 09:31:49,365 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,365 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,365 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:31:49,409 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:49,438 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:49,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,455 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,455 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:31:49,605 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,605 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,605 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:31:49,695 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,695 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,695 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:31:49,756 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:49,756 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,756 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,811 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:49,864 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,864 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,864 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:31:49,864 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:49,865 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,866 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:49,867 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:49,867 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:49,867 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 09:31:49,867 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:31:49,868 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:49,868 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 09:31:49,868 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 09:31:49,868 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:49,868 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:49,869 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:49,869 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:49,869 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 09:31:49,869 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:49,869 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:49,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:31:49,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,974 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:31:49,974 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,976 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:49,977 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:31:49,978 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:31:49,978 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:49,978 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:49,978 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:49,978 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 09:31:49,978 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 09:31:49,978 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 09:31:49,978 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:49,978 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:31:49,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:49,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:49,978 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 09:31:49,979 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 09:31:49,979 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:49,979 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:49,985 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 09:31:50,49 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#-877532440] 09:31:50,49 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 09:31:50,49 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493] 09:31:50,50 AM [main] [DEBUG] TransactionProxy - New WRITE_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,50 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1365243954] 09:31:50,50 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:50,50 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1365243954] 09:31:50,50 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:50,50 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying 2 components for commit 09:31:50,50 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard cars 09:31:50,50 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:50,50 AM [main] [DEBUG] TransactionProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Readying transaction for shard people 09:31:50,50 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Queuing TransactionOperation 09:31:50,50 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 canCommit 09:31:50,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:50,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:50,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:31:50,50 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:50,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - creating WRITE_ONLY transaction context 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 executeWrite called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionContext - Total modifications on Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 = [ 1 ] 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction called 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 sending 1 batched modifications, ready: true 09:31:50,51 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.BatchedModifications to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: encountered ask-based client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, disabling transaction tracking 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:31:50,53 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:50,53 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:50,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 09:31:50,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying 1 batched modifications for Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,54 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 12 09:31:50,54 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,54 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 09:31:50,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Readying Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 of 1 operations, client version 12 09:31:50,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,54 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 ready 09:31:50,56 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,56 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}]]] 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:31:50,58 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:50,58 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074)] resolved 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransactionReadyReplyMapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 readyTransaction 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: cohort actor ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493)] resolved 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: successfully resolved all cohort actors 09:31:50,58 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finishCanCommit 09:31:50,59 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 modifications completed with ReadyTransactionReply [version=12] 09:31:50,59 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,59 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074)] 09:31:50,59 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65}] 09:31:50,59 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074)] 09:31:50,59 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=6600624} is already disabled 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader state Disabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=6243098} is already disabled 09:31:50,59 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:50,59 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:50,60 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:50,60 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, size=65} 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: disabling tracking of ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: disableTracking ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} does not match any client, pre-disabling client. 09:31:50,60 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:50,60 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:31:50,61 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:31:50,61 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: sending CanCommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493)] 09:31:50,61 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493)] 09:31:50,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Can committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 validated 09:31:50,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:50,61 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: canCommit for member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: success 09:31:50,62 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:31:50,62 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit returning result: true 09:31:50,62 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 commit 09:31:50,62 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 finish commit 09:31:50,62 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074)] 09:31:50,62 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-2046665074)] 09:31:50,62 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: Sending CommitTransaction [transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493)] 09:31:50,62 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#-2074945493)] 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Committing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 candidate ready 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: canCommit - messages: [] 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 175 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 prepared 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Finishing commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 179 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:50,63 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 09:31:50,63 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 submitted to persistence 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}]]] 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=175} 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}]]] 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=179} 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 09:31:50,64 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:50,64 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 09:31:50,65 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=179}] 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=175}] 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-cars-testWriteTransactionWithMultipleShards%23-2046665074$e] 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testWriteTransactionWithMultipleShards_member-1-shard-people-testWriteTransactionWithMultipleShards%23-2074945493$f] 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#91619818] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-2089037586] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:50,65 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:50,65 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:50,66 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:50,66 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-2089037586] 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#91619818] 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 79.23 μs 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 47.45 μs 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-2089037586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,66 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#91619818] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=179} 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 1 data Simple{size=175} 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1310964335] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0: commit succeeded 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:50,67 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#243810794] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 09:31:50,67 AM [main] [DEBUG] TransactionProxy - New READ_ONLY Tx - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1310964335] 09:31:50,67 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] for shard cars, trying to use DirectTransactionContextWrapper 09:31:50,67 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:50,67 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:50,67 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:50,67 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards)] 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:50,67 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-1310964335] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 34.90 μs 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#243810794] 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#243810794] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,67 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 29.28 μs 09:31:50,68 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:50,69 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_28#855635710, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=12] 09:31:50,69 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars 09:31:50,69 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_28#855635710)] 09:31:50,73 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_28#855635710] 09:31:50,73 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_28#855635710] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,74 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 09:31:50,74 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Found primary ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] for shard people, trying to use DirectTransactionContextWrapper 09:31:50,74 AM [main] [DEBUG] AbstractTransactionContextFactory - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0: Local transaction context creation failed, using DelayedTransactionWrapper 09:31:50,74 AM [main] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Primary shard ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] found - trying create transaction 09:31:50,74 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CreateTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards)] 09:31:50,74 AM [main] [DEBUG] DelayedTransactionContextWrapper - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Queuing TransactionOperation 09:31:50,75 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Creating transaction : member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:31:50,75 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardReadTransaction - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_29#1464000450] 09:31:50,76 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContextSupport - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 Received CreateTransactionReply [transactionPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_29#1464000450, transactionId=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0, version=12] 09:31:50,76 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 executeRead ReadData called path = /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people 09:31:50,76 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.ReadData to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_29#1464000450)] 09:31:50,77 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 ReadData operation succeeded 09:31:50,77 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,78 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,78 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:50,78 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 09:31:50,78 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 09:31:50,78 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onShutDown 09:31:50,78 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,78 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 09:31:50,78 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:50,78 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:31:50,78 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 09:31:50,78 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:50,78 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:50,78 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:31:50,78 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:31:50,78 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 09:31:50,79 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1365243954] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1365243954] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cd92700 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5cd92700 09:31:50,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:31:50,79 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51dac1f8 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@51dac1f8 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:31:50,79 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:50,79 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testWriteTransactionWithMultipleShards: Failed to transfer leadership in 255.7 ms 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In unpauseLeader 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-782: Closing 09:31:50,335 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testWriteTransactionWithMultipleShards: Failed to transfer leadership in 265.6 ms 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In unpauseLeader 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:31:50,345 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-781: Closing 09:31:50,346 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:31:50,346 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:50,346 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:31:50,346 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:31:50,346 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 09:31:50,346 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:31:50,347 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:50,347 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:50,354 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:50,354 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:50,354 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:50,354 AM [cluster-test-akka.actor.default-dispatcher-13] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:50,355 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:50,366 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:50,366 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:51,360 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8390208602871671502] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:51,381 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2318459049985395869] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:51,382 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:51,382 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:51,384 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:51.385] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:51,390 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:51,390 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:51,393 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:51.394] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:51,414 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:51,421 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4503145731078874685] 09:31:51,421 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:51,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:51,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:51,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:51,428 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:51,431 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:51,431 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:51,431 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:51,438 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:51,443 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [2738302014746007285] 09:31:51,443 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:51,444 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:51,444 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:51,453 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:51,453 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:51,467 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:51,482 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1232018830878212549] 09:31:51,482 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:51,484 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:51,484 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:51,497 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:51,498 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:51,500 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 09:31:51,510 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 09:31:51,510 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1392119676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:51,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:51,574 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:51,577 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:51,584 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 09:31:51,584 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,584 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,585 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:51,585 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,585 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,585 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 09:31:51,585 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:51,587 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@26447385 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54177de6 09:31:51,587 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1115f900 09:31:51,591 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1714037511588, None)) 09:31:51,591 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037511587,0,0)) 09:31:51,591 AM [cluster-test-akka.actor.default-dispatcher-15] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54177de6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4f592b5d 09:31:51,591 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:51,591 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5d76e2f9 09:31:51,591 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 09:31:51,592 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:51,593 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:51,594 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:51,594 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-1-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769] 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithMultipleShards, persistent : true 09:31:51,595 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 09:31:51,595 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionChainWithMultipleShards, persistent : true 09:31:51,595 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,595 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-1-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631] 09:31:51,596 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1438864644 created and ready for shard:member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-notifier#1438864644] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1960509320] 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#1960509320] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1999730769] 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-2485777] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,596 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/shard-member-1:snapshot-read#-1999730769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,597 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-notifier#-2485777 created and ready for shard:member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,597 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithMultipleShards 09:31:51,599 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2054313825] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:51,599 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 09:31:51,599 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionChainWithMultipleShards is initialized 09:31:51,599 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,599 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:51,599 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 09:31:51,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithMultipleShards is initialized 09:31:51,599 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:31:51,600 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithMultipleShards 09:31:51,600 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,600 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,600 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 09:31:51,600 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:31:51,602 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from null to Follower 09:31:51,602 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from null to Follower 09:31:51,603 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithMultipleShards 09:31:51,603 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:51,604 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Creating local shard: member-2-shard-people-testTransactionChainWithMultipleShards 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: no initial generation override, starting from 0 09:31:51,604 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:51,604 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,604 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1f09474d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@756c87d9 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@71530dac 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithMultipleShards, 0, 1714037511604, None)) 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037511603,0,0)) 09:31:51,604 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@babf0e8 09:31:51,604 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@756c87d9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@146c9032 09:31:51,604 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} started 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-cars-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards on shard member-2-shard-cars-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#-517368058] 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:51,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: updated peer member-1-shard-people-testTransactionChainWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards on shard member-2-shard-people-testTransactionChainWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#1394084526] 09:31:51,605 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithMultipleShards, persistent : true 09:31:51,605 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards#-517368058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionChainWithMultipleShards, persistent : true 09:31:51,606 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithMultipleShards with journal batch size 1 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards#1394084526] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#818438754] 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-1352298877 created and ready for shard:member-2-shard-people-testTransactionChainWithMultipleShards 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#996792110] 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#165057943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#818438754] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-notifier#-1352298877] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/shard-member-2:snapshot-read#996792110] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:51,606 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:51,606 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:51,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,607 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [cars] 09:31:51,607 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,607 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-notifier#165057943 created and ready for shard:member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithMultipleShards is initialized 09:31:51,607 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , received role change from null to Follower 09:31:51,607 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:31:51,607 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionChainWithMultipleShards with journal batch size 1 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-cars-testTransactionChainWithMultipleShards from null to Follower 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:51,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Initializing shard [people] 09:31:51,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionChainWithMultipleShards is initialized 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,607 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,607 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , received role change from null to Follower 09:31:51,608 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,608 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionChainWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards 09:31:51,608 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-2-shard-people-testTransactionChainWithMultipleShards from null to Follower 09:31:51,655 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1392119676] 09:31:51,655 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : cars 09:31:51,655 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769] 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,825 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 09:31:51,825 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Follower to Candidate 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 1 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:31:51,855 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionChainWithMultipleShards is no longer the leader 09:31:51,855 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Follower to Candidate 09:31:51,856 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Follower to Candidate 09:31:52,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,85 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 09:31:52,105 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,105 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,105 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 2 09:31:52,134 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:52,203 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:52,355 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,355 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,355 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 09:31:52,395 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,395 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,395 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 3 09:31:52,436 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:52,436 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:52,436 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:52,436 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 09:31:52,436 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,437 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionChainWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,446 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:52,446 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards 09:31:52,446 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithMultipleShards/member-3-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,447 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionChainWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards to akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,499 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:52,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,645 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 09:31:52,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,650 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:52,653 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:52,653 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:52,653 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionChainWithMultipleShards] 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: became leader with frontend state for [] 09:31:52,654 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:52,654 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-people-testTransactionChainWithMultipleShards from Candidate to Leader 09:31:52,654 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards: Closing 09:31:52,654 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,655 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionChainWithMultipleShards, leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:31:52,655 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:52,655 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:52,655 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done false 09:31:52,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:52,655 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:52,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Received ElectionTimeout 09:31:52,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,694 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Starting new election term 4 09:31:52,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Cluster state: [] 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=null 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,696 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithMultipleShards] 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: became leader with frontend state for [] 09:31:52,697 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithMultipleShards , received role change from Candidate to Leader 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received role changed for member-1-shard-cars-testTransactionChainWithMultipleShards from Candidate to Leader 09:31:52,697 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards: Closing 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithMultipleShards, leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderPayloadVersion=12] 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All Shards are ready - data store testTransactionChainWithMultipleShards is ready 09:31:52,700 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done false 09:31:52,702 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:31:52,703 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:52,762 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1392119676] 09:31:52,762 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: findLocalShard : people 09:31:52,762 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631] 09:31:52,762 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 09:31:52,762 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from IDLE to TX_OPEN 09:31:52,763 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 09:31:52,763 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:31:52,763 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:31:52,763 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2054313825] 09:31:52,763 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:52,763 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 09:31:52,763 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:31:52,763 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=12] 09:31:52,763 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:31:52,763 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:31:52,763 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards, primaryVersion=12] 09:31:52,763 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 09:31:52,763 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:31:52,763 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:31:52,763 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#2054313825] 09:31:52,763 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@263451a3, attempting to connect 09:31:52,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:31:52,764 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 09:31:52,764 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=12] 09:31:52,764 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:31:52,764 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards, primaryVersion=12] 09:31:52,764 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 09:31:52,764 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@513504f6, attempting to connect 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@7764b8fe 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@26453add 09:31:52,764 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from TX_OPEN to IDLE 09:31:52,764 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@4a4284f9 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 09:31:52,764 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@3445b19e 09:31:52,768 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-people-testTransactionChainWithMultipleShards status sync done true 09:31:52,775 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 09:31:52,775 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=9950, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:52,777 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} is not yet known 09:31:52,777 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, nanosAgo=5810, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:52,779 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:31:52,779 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@212658b3 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719056885237} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 09:31:52,779 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@43aa1779 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}, enqueuedTicks=719057755272} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 409 bytes, total slices to send: 1 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@112dcb2e 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719057973295} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 289 bytes, total slices to send: 1 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5f7e45e0 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56731bc6 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7e611c13, splicing queue 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@56731bc6 queue spliced 0 messages 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.674 ms 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:31:52,780 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7922a92d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719057471097} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@17c612f2 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}, enqueuedTicks=719057813123} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 417 bytes, total slices to send: 1 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@376c0a97 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719058058327} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 290 bytes, total slices to send: 1 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@647ff69 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5f45bf00 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2d3d8a23, splicing queue 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@5f45bf00 queue spliced 0 messages 09:31:52,781 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 601.6 μs 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:31:52,785 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@73819cc9 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,785 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@20e58cc4 09:31:52,786 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,786 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,786 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 ready 09:31:52,786 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 ready 09:31:52,786 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:31:52,786 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 09:31:52,786 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 09:31:52,786 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 to ready 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 is already in state READY (READY) 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating canCommit 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 validated 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 validated 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed canCommit 09:31:52,787 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}, enqueuedTicks=719057755272, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY}, enqueuedTicks=719057813123, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719057973295, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 09:31:52,791 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719058058327, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1}} 09:31:52,792 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@69280169 09:31:52,792 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@379335f9 09:31:52,792 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719085595380} 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@2ec3bbd4 09:31:52,792 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 09:31:52,792 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@6a56b3e0 09:31:52,792 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719085778253} 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating preCommit 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating preCommit 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 candidate ready 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 candidate ready 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - candidate rootPath: / 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: canCommit - messages: [] 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 prepared 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: canCommit - messages: [] 09:31:52,794 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed preCommit 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: preCommit - successfulFromPrevious: [] 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 prepared 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:52,794 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed preCommit 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719085595380, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719085778253, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2}} 09:31:52,796 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:31:52,796 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6d4e0df5 09:31:52,797 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 3 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6d4e0df5 09:31:52,797 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719090418203} 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@5ee00693 09:31:52,797 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 3 09:31:52,797 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@5ee00693 09:31:52,797 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719090563356} 09:31:52,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 initiating commit 09:31:52,798 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 initiating commit 09:31:52,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,798 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 177 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 181 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 09:31:52,799 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 submitted to persistence 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 09:31:52,799 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 submitted to persistence 09:31:52,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 09:31:52,800 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}]]] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=177} 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}]]] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=181} 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:31:52,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=177}] 09:31:52,800 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards Received follower initial sync status for member-2-shard-cars-testTransactionChainWithMultipleShards status sync done true 09:31:52,800 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,801 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=181}] 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 0 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2178/0x000000080169b4b8@22195492 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2178/0x000000080169b4b8@29f4cb79 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] LeaderFrontendState - member-1-shard-people-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6f8009fd}, purged=MutableUnsignedLongSet{size=0}} 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@575aa5cb}, purged=MutableUnsignedLongSet{size=0}} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 1 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2: commit - successfulFromPrevious: [] 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE, modificationType=WRITE}}}]} 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-78907039] with name member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2040092579] with name member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:52,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2040092579] 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:52,801 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:52,801 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:31:52,801 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 54.48 μs 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-2040092579] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-78907039] 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 28.86 μs 09:31:52,802 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#-78907039] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719057471097, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719056885237, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719090563356, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3}} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} doCommit completed 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7922a92d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 4 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7c3fd74f 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2} 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 4 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@393d46ac 09:31:52,804 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719098111636} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719090418203, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3}} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} doCommit completed 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@212658b3 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 4 09:31:52,805 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from IDLE to TX_OPEN 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@8cebdd9 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 4 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@a499620 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:31:52,805 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719098440661} 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 0 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68} 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from -1 to 0 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68} 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7143ca97}, purged=MutableUnsignedLongSet{size=0}} 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,805 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=181} 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4046a8dc}, purged=MutableUnsignedLongSet{size=0}} 09:31:52,805 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 0 to 1 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 1 data Simple{size=177} 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,805 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:31:52,805 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 0 09:31:52,805 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,805 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@35554c19 09:31:52,805 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}, enqueuedTicks=719099162003} 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1654910796] with name member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher 09:31:52,805 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}} 09:31:52,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:52,805 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:52,806 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-888718149] with name member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-888718149] 09:31:52,806 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:31:52,806 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:31:52,806 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 453 bytes, total slices to send: 1 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,806 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 42.14 μs 09:31:52,806 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@52935944 09:31:52,806 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=719099586021} 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards/member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher#-888718149] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1654910796] 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 22.16 μs 09:31:52,806 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards/member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher#1654910796] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}]]] 09:31:52,810 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,810 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,810 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,810 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:31:52,810 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@7b1aed7d 09:31:52,811 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 09:31:52,811 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 09:31:52,812 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}]]] 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 1 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 09:31:52,812 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 09:31:52,812 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}, enqueuedTicks=719099162003, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0}} 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 2 09:31:52,812 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=0} 09:31:52,812 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,813 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: Message size: 371 bytes, total slices to send: 1 09:31:52,813 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: Message does not need to be sliced - sending original message 09:31:52,814 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69}] 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 2, nextIndex: 3 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@1fe5cd90 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:31:52,814 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:31:52,815 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 09:31:52,815 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=719099586021, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1}} 09:31:52,815 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=1} 09:31:52,815 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 1 09:31:52,815 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@2f462cc5 09:31:52,815 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}, enqueuedTicks=719108565395} 09:31:52,815 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}} 09:31:52,815 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 441 bytes, total slices to send: 1 09:31:52,815 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,815 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@2ae05785 09:31:52,815 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=719108925681} 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 2 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, size=69} 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719098440661, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=4}} 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 09:31:52,816 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@212658b3 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-1 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 1 for term 4 09:31:52,816 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=2 09:31:52,821 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,821 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,821 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@12975922 09:31:52,822 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 09:31:52,822 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: Message size: 358 bytes, total slices to send: 1 09:31:52,822 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: Message does not need to be sliced - sending original message 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1}, enqueuedTicks=719108565395, sessionId=1, txSequence=6} with Envelope{sessionId=1, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0}} 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=0} 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=719108925681, sessionId=1, txSequence=7} with Envelope{sessionId=1, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1}} 09:31:52,825 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=1} 09:31:52,827 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 2 09:31:52,827 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@2342685b 09:31:52,827 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:31:52,827 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}, enqueuedTicks=719121118811} 09:31:52,827 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}} 09:31:52,827 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 290 bytes, total slices to send: 1 09:31:52,827 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} allocated sequence 2 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@5db73783 09:31:52,828 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}, enqueuedTicks=719121333184} 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}} 09:31:52,828 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 291 bytes, total slices to send: 1 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,828 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from TX_OPEN to IDLE 09:31:52,828 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:31:52,828 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from IDLE to TX_OPEN 09:31:52,828 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} allocated sequence 0 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@b98d67f 09:31:52,828 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY}, enqueuedTicks=719121671810} 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY}} 09:31:52,828 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 427 bytes, total slices to send: 1 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,828 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:31:52,828 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from TX_OPEN to IDLE 09:31:52,828 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 readied transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 3 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@7cd0bbd4 09:31:52,828 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc skipping transactions [2] 09:31:52,828 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719121993646} 09:31:52,828 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719122040076} 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}} 09:31:52,828 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 290 bytes, total slices to send: 1 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 3 09:31:52,828 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@19551d9f 09:31:52,828 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719122159018} 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, message: Envelope{sessionId=1, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}} 09:31:52,828 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 291 bytes, total slices to send: 1 09:31:52,828 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 ready 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 to ready 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 ready 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:52,831 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 to ready 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}] 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: readying transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 ready 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE}}}}}}}]} 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: transitioned member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 to ready 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 is already in state READY (READY) 09:31:52,832 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating canCommit 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 validated 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed canCommit 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipping transactions [[2..2]] 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.SkipTransactionsPayload, isSendImmediate: false 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 is already in state READY (READY) 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating canCommit 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 validated 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:52,833 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed canCommit 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}, enqueuedTicks=719121333184, sessionId=1, txSequence=8} with Envelope{sessionId=1, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2}} 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=2} 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY}, enqueuedTicks=719121118811, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2}} 09:31:52,834 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=2} 09:31:52,836 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY}, enqueuedTicks=719121671810, sessionId=1, txSequence=9} with Envelope{sessionId=1, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=0} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719122159018, sessionId=1, txSequence=10} with Envelope{sessionId=1, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=3}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} canCommit completed 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=719122040076, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=3}} 09:31:52,836 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} canCommit completed 09:31:52,837 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 4 09:31:52,837 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@3ff0405d 09:31:52,837 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719131104912} 09:31:52,837 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 4 09:31:52,837 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@41e3c613 09:31:52,837 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719131169473} 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating preCommit 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 candidate ready 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: canCommit - messages: [] 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 prepared 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 completed preCommit 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating preCommit 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 candidate ready 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - candidate rootPath: / 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: canCommit - messages: [] 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: preCommit - successfulFromPrevious: [] 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 prepared 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:52,838 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed preCommit 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719131104912, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=4}} 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} preCommit completed, clearing successfulRequests 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719131169473, sessionId=1, txSequence=11} with Envelope{sessionId=1, txSequence=b, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=4}} 09:31:52,839 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} preCommit completed, clearing successfulRequests 09:31:52,839 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 5 09:31:52,839 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@31b767e0 09:31:52,839 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719132942054} 09:31:52,839 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} allocated sequence 5 09:31:52,839 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@cf82b23 09:31:52,839 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719132991705} 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 initiating commit 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 236 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 submitted to persistence 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 initiating commit 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 225 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 2 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 09:31:52,840 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 submitted to persistence 09:31:52,841 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 09:31:52,841 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}]]] 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=236} 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 09:31:52,843 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Log size is now 3 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithMultipleShards, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}]]] 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=225} 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 09:31:52,844 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,844 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75}] 09:31:52,844 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=Simple{size=236}] 09:31:52,844 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69}] 09:31:52,844 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=225}] 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithMultipleShards updated: matchIndex: 4, nextIndex: 5 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 2 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 3, nextIndex: 4 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 2 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2186/0x00000008016a4658@6c71e93 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@7c2e0e84 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 3 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2: commit - successfulFromPrevious: [] 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 09:31:52,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 09:31:52,845 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:31:52,846 AM [cluster-test-shard-dispatcher-32] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:31:52,846 AM [cluster-test-shard-dispatcher-33] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:31:52,846 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 09:31:52,846 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 09:31:52,846 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:31:52,846 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:31:52,846 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 09:31:52,846 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 09:31:52,846 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,846 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 56.22 μs 09:31:52,846 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 76.93 μs 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 3 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 09:31:52,847 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Commit index set to 4 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719098111636, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=4}} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE}: purge completed 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: purge completed 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 2 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7922a92d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 1 to 2 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2, size=69} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=SkipTransactionsRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719121993646, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-1, sequence=0}} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc confirmed transaction skip 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719132942054, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=5}} 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-2 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} doCommit completed 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED} 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 3 data Simple{size=225} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 6 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@bddf55f 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-cars-testTransactionChainWithMultipleShards#1738786769], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719141146205} 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 09:31:52,847 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,847 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719132991705, sessionId=1, txSequence=12} with Envelope{sessionId=1, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=5}} 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} doCommit completed 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED} 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=FLUSHED}: initiating purge 09:31:52,848 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} allocated sequence 1 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} allocated sequence 6 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4097fb0e 09:31:52,848 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@144f420d 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719141344488} 09:31:52,848 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=SIMPLE}, enqueuedTicks=719141394209} 09:31:52,848 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, message: Envelope{sessionId=1, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=SIMPLE}} 09:31:52,848 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 291 bytes, total slices to send: 1 09:31:52,848 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:52,848 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,848 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 2 for term 4 09:31:52,848 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 38.72 μs 09:31:52,848 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=3 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 3 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 2 to 3 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 3 data SkipTransactionsPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=75} 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: History LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} skipped transactions ImmutableUnsignedLongSet{span=[2..2], size=1} 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying state for log index 4 data Simple{size=236} 09:31:52,848 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,849 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,849 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,849 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithMultipleShards-DTCL-publisher: 23.16 μs 09:31:52,849 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 3 for term 4 09:31:52,849 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=4 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 09:31:52,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 09:31:52,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 09:31:52,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:31:52,854 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 is already in state READY (READY) 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct canCommit 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Validating transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 validated 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct preCommit 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Preparing transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 candidate ready 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - candidate rootPath: / 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: canCommit - messages: [] 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: preCommit - successfulFromPrevious: [] 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 prepared 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 initiating direct commit 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Starting commit for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 222 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 4, leaderLastIndex: 5, leaderSnapShotIndex: 2 09:31:52,854 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 4 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,855 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 09:31:52,855 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 submitted to persistence 09:31:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 09:31:52,856 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}] 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}], SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}]]] 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Number of entries to be appended = 2 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Append entry to log Simple{size=222} 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Log size is now 3 09:31:52,857 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionChainWithMultipleShards, logLastIndex=5, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionChainWithMultipleShards updated: matchIndex: 5, nextIndex: 6 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 3 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 4 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: replication of member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@281ca6e9 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: finished purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Setting last applied to 5 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Resuming commit of transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2: commit - successfulFromPrevious: [] 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person, operation=TOUCH, modificationType=DISAPPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}], operation=DELETE, modificationType=DELETE}}}}}}}]} 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:31:52,857 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 09:31:52,858 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,858 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 42.15 μs 09:31:52,858 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70}] 09:31:52,858 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=Simple{size=222}] 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719141344488, sessionId=1, txSequence=13} with Envelope{sessionId=1, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, sequence=6}} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE}: purge completed 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, state=DONE} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=SIMPLE} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=SIMPLE}, enqueuedTicks=719141394209, sessionId=1, txSequence=14} with Envelope{sessionId=1, txSequence=e, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=1}} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED} directCommit completed 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=FLUSHED}: initiating purge 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} allocated sequence 2 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@1ba0dad 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithMultipleShards/member-1-shard-people-testTransactionChainWithMultipleShards#70707631], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:31:52,861 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719154909591} 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Commit index set to 5 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 3, lastIndex(): 5 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 4 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 3 to 4 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2, size=70} 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Purged transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-2 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Setting last applied to 5 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Moving last applied index from 4 to 5 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Applying state for log index 5 data Simple{size=222} 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionChainWithMultipleShards: Applying foreign transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionChainWithMultipleShards: Committed transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,862 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionChainWithMultipleShards: publishChanges: rootPath: / 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: fakeSnapshot purging log to 4 for term 4 09:31:52,862 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Persisting ApplyJournalEntries with index=5 09:31:52,862 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionChainWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testTransactionChainWithMultipleShards-DTCL-publisher: 62.53 μs 09:31:52,862 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-0 09:31:52,862 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-0 09:31:52,862 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-0-0 09:31:52,862 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@59d5e2dc invoking destroy 09:31:52,862 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719156134842} 09:31:52,862 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@27cc4132 invoking destroy 09:31:52,862 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504]}, enqueuedTicks=719156194733} 09:31:52,862 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@670e0fa2 changed state from IDLE to CLOSED 09:31:52,862 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@20660ce6: creating a new snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:31:52,863 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 09:31:52,863 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:31:52,863 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:31:52,863 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@63a3a2ff 09:31:52,863 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=719156409487} 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 09:31:52,875 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: purging transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,876 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 09:31:52,877 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:31:52,877 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: Message size: 370 bytes, total slices to send: 1 09:31:52,877 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards_slicer-id-784: Message does not need to be sliced - sending original message 09:31:52,877 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}] 09:31:52,877 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}] 09:31:52,877 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}] 09:31:52,879 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:31:52,879 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=719156409487, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:31:52,879 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=0} 09:31:52,879 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 09:31:52,879 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0 09:31:52,879 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:31:52,879 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@5a3b3d5 09:31:52,879 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=719172940921} 09:31:52,880 AM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionChainWithMultipleShards: allocating new open snapshot member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2 09:31:52,880 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:31:52,880 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: Message size: 232 bytes, total slices to send: 1 09:31:52,880 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards_slicer-id-785: Message does not need to be sliced - sending original message 09:31:52,881 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:31:52,881 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-328140504], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}]}, enqueuedTicks=719172940921, sessionId=1, txSequence=17} with Envelope{sessionId=1, txSequence=11, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:31:52,881 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person/person[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)name=jack}] completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=0} 09:31:52,881 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:52,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,881 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 09:31:52,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: onShutDown 09:31:52,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: onShutDown 09:31:52,882 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,882 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithMultipleShards (Follower): Closing 09:31:52,882 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:31:52,882 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionChainWithMultipleShards (Follower): Closing 09:31:52,882 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:31:52,882 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:52,882 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 09:31:52,883 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:31:52,883 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithMultipleShards, generation=0} 09:31:52,883 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 09:31:52,883 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:31:52,883 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-328140504] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-328140504] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:52,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:52,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testTransactionChainWithMultipleShards 09:31:52,885 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithMultipleShards - waiting on 2 shards 09:31:52,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: onShutDown 09:31:52,885 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: onShutDown 09:31:52,885 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Initiating leader transfer 09:31:52,885 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Initiating leader transfer 09:31:52,886 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:52,886 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1bbeee3e 09:31:52,886 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6cafe731 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1} 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1bbeee3e 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2} 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionChainWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@6cafe731 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Attempting to transfer leadership 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testTransactionChainWithMultipleShards 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionChainWithMultipleShards: active: true, followerNextIndex: 6, leaderLastIndex: 7, leaderSnapShotIndex: 4 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:31:52,886 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithMultipleShards, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-1-1, size=70}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=1}, size=68}]]] 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionChainWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionChainWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:31:52,886 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:31:53,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): sendAppendEntries: 6 is present for follower member-2-shard-people-testTransactionChainWithMultipleShards 09:31:53,5 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionChainWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testTransactionChainWithMultipleShards, prevLogIndex=5, prevLogTerm=4, leaderCommit=5, replicatedToAllIndex=4, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=6, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-chn-1-txn-2-2, size=70}], SimpleReplicatedLogEntry [index=7, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithMultipleShards, generation=0}, history=1, cookie=2}, size=68}]]] 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testTransactionChainWithMultipleShards: Failed to transfer leadership in 239.1 ms 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: In unpauseLeader 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards (Leader): Leadership transfer expired 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionChainWithMultipleShards 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionChainWithMultipleShards (Leader)_slicer-id-789: Closing 09:31:53,125 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-people-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:31:53,175 AM [cluster-test-shard-dispatcher-32] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionChainWithMultipleShards: Failed to transfer leadership in 289.5 ms 09:31:53,175 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: In unpauseLeader 09:31:53,175 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: leader transfer failed - sending PoisonPill 09:31:53,175 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader): Leadership transfer expired 09:31:53,175 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithMultipleShards 09:31:53,176 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithMultipleShards (Leader)_slicer-id-790: Closing 09:31:53,176 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithMultipleShards: Aborting 0 pending queued transactions 09:31:53,176 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionChainWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:31:53,176 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithMultipleShards 09:31:53,176 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:31:53,176 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithMultipleShards: shutting down 09:31:53,176 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithMultipleShards: Closing 09:31:53,177 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1392119676] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithMultipleShards#-1392119676] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:53,177 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:53,177 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:53,180 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:53,181 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:53,181 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:53,181 AM [cluster-test-akka.actor.default-dispatcher-27] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:53,186 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:53,186 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:54,188 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4503145731078874685] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:54,206 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:54,206 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:54,208 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:54.209] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:54,210 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [2738302014746007285] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:54,225 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:54,225 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:31:54.230] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:54,262 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:54,268 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8965649875919904103] 09:31:54,269 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:54,270 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:54,270 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:54,270 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:54,274 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:54,279 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:54,279 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:54,280 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:54,289 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@666558e2 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} 09:31:54,289 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:31:54,289 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@1cc87ec3 09:31:54,289 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@58aebab8 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT}, enqueuedTicks=720582439269} 09:31:54,289 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT}} 09:31:54,289 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 288 bytes, total slices to send: 1 09:31:54,289 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:54,289 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3eb0c1d0 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} 09:31:54,289 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 1 09:31:54,289 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@4ebe2cc1 09:31:54,289 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@692fd326 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT}, enqueuedTicks=720582744994} 09:31:54,289 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: slice: identifier: member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=12, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-328140504], modifications=0, protocol=ABORT}} 09:31:54,289 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message size: 289 bytes, total slices to send: 1 09:31:54,289 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithMultipleShards_slicer-id-786: Message does not need to be sliced - sending original message 09:31:54,289 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithMultipleShards-fe-0-txn-0-0} 09:31:54,290 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 09:31:54,290 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-people-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_29#1464000450)] 09:31:54,290 AM [com.google.common.base.internal.Finalizer] [DEBUG] RemoteTransactionContext - Tx member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 closeTransaction called 09:31:54,290 AM [com.google.common.base.internal.Finalizer] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CloseTransaction to ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-cars-member-2:datastore-testWriteTransactionWithMultipleShards@0:1_28#855635710)] 09:31:54,292 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:54,296 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [793452295753815687] 09:31:54,296 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:54,298 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:54,298 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:54,298 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:54,298 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:54,314 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:54,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [3538409224086246027] 09:31:54,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:54,321 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:54,321 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:54,322 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:54,322 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:54,330 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:54,332 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:54,333 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 09:31:54,337 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 09:31:54,340 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 09:31:54,343 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 09:31:54,343 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 09:31:54,343 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-1-shard-cars-testInstallSnapshot 09:31:54,343 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@29c7306b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d2c5b09 09:31:54,343 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@282ed8c8 09:31:54,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:54,343 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,343 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testInstallSnapshot, 0, 1714037514343, None)) 09:31:54,344 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037514342,0,0)) 09:31:54,344 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@43c3da57 09:31:54,344 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6d2c5b09 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4db742fa 09:31:54,344 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} started 09:31:54,345 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:54,346 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:54,348 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testInstallSnapshot 09:31:54,348 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:54,348 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testInstallSnapshot 09:31:54,348 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:54,349 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,349 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,349 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-1-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653] 09:31:54,349 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testInstallSnapshot, persistent : true 09:31:54,350 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testInstallSnapshot with journal batch size 1 09:31:54,351 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,352 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-70466418 created and ready for shard:member-1-shard-cars-testInstallSnapshot 09:31:54,352 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1873099452] 09:31:54,352 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-notifier#-70466418] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,352 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/shard-member-1:snapshot-read#-1873099452] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,352 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.SnapshotOffer from Actor[akka://cluster-test/system/in-memory-snapshot-store#-3975753] 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotOffer called. 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=1, votedFor=null 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Applying recovered snapshot 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testInstallSnapshot: clearing clients {} 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-1365244431] with name member-1-shard-cars-testInstallSnapshot-DTCL-publisher 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: state snapshot applied in 428.2 μs 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Recovery snapshot applied for member-1-shard-cars-testInstallSnapshot in 520.6 μs: snapshotIndex=5, snapshotTerm=1, journal-size=0 09:31:54,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-1365244431] 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:54,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 09:31:54,353 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Recovery completed in 818.0 μs - Switching actor to Follower - last log index = 5, last log term = 1, snapshot index = 5, snapshot term = 1, journal size = 0 09:31:54,353 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot-DTCL-publisher#-1365244431] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,354 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:31:54,354 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 09:31:54,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-1-shard-cars-testInstallSnapshot-DTCL-publisher: 98.69 μs 09:31:54,354 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from null to Follower 09:31:54,354 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 09:31:54,354 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testInstallSnapshot is initialized 09:31:54,354 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 09:31:54,354 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from null to Follower 09:31:54,355 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:54,356 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:54,361 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1670712933] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,362 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testInstallSnapshot 09:31:54,365 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: no initial generation override, starting from 0 09:31:54,365 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 09:31:54,365 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@254c0735 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fa967da 09:31:54,366 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@13ecb1d2 09:31:54,366 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Recovery complete : shard-manager-testInstallSnapshot 09:31:54,366 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testInstallSnapshot, 0, 1714037514366, None)) 09:31:54,366 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Creating local shard: member-2-shard-cars-testInstallSnapshot 09:31:54,366 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:54,366 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,367 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testInstallSnapshot: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037514365,0,0)) 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79bc0226 09:31:54,367 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fa967da to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@53e5300a 09:31:54,367 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} started 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testInstallSnapshot 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:54,367 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: updated peer member-1-shard-cars-testInstallSnapshot on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot on shard member-2-shard-cars-testInstallSnapshot whose actor address is Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#723076967] 09:31:54,368 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testInstallSnapshot, persistent : true 09:31:54,368 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot#723076967] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1243576507 created and ready for shard:member-2-shard-cars-testInstallSnapshot 09:31:54,368 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-notifier#-1243576507] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,368 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#2105707152] 09:31:54,368 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testInstallSnapshot with journal batch size 1 09:31:54,368 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/shard-member-2:snapshot-read#2105707152] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:54,368 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:54,369 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:54,369 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Initializing shard [cars] 09:31:54,369 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-2-shard-cars-testInstallSnapshot is initialized 09:31:54,369 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:31:54,369 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testInstallSnapshot is no longer the leader 09:31:54,369 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:54,369 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , received role change from null to Follower 09:31:54,369 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testInstallSnapshot , registered listener akka://cluster-test/user/shardmanager-testInstallSnapshot 09:31:54,369 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-2-shard-cars-testInstallSnapshot from null to Follower 09:31:54,417 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] 09:31:54,418 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: findLocalShard : cars 09:31:54,418 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653] 09:31:54,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:54,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Follower): Received ElectionTimeout - switching to Candidate 09:31:54,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testInstallSnapshot] 09:31:54,626 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=2, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 2 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Follower) :- Switching from behavior Follower to Candidate, election term: 2 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testInstallSnapshot (Follower): Closing 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: false, hasLeader : false 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testInstallSnapshot is no longer the leader 09:31:54,627 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Follower to Candidate 09:31:54,627 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Follower to Candidate 09:31:54,866 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:31:54,866 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=3, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:54,866 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 3 09:31:54,939 AM [cluster-test-shard-dispatcher-23] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:55,116 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:31:55,116 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=4, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:55,116 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 4 09:31:55,286 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:55,286 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:55,287 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Peer address for peer member-2-shard-cars-testInstallSnapshot set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testInstallSnapshot/member-3-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:55,293 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testInstallSnapshot with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot to akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot 09:31:55,339 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:55,346 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Received ElectionTimeout 09:31:55,346 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:55,346 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Starting new election term 5 09:31:55,346 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): Sending RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] to peer member-2-shard-cars-testInstallSnapshot 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Cluster state: [] 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] with higher term appears reachable, updating term. 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Term 5 in "RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1]" message is greater than follower's term 0 - updating term 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=null 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): In requestVote: RequestVote [term=5, candidateId=member-1-shard-cars-testInstallSnapshot, lastLogIndex=5, lastLogTerm=1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:55,347 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:31:55,352 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:31:55,352 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:31:55,352 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Election: Leader has following peers: [member-2-shard-cars-testInstallSnapshot] 09:31:55,352 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: false, followerNextIndex: 5, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:31:55,353 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:31:55,353 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testInstallSnapshot: onLeadershipChange, isLeader: true, hasLeader : true 09:31:55,353 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: became leader with frontend state for [] 09:31:55,353 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot: Closing 09:31:55,353 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: getLeaderAddress leaderId = member-1-shard-cars-testInstallSnapshot peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot 09:31:55,353 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done false 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testInstallSnapshot updated: matchIndex: -1, nextIndex: 0 09:31:55,354 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=12] 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 0, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:31:55,354 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:55,354 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:55,354 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Initiating install snapshot to follower member-2-shard-cars-testInstallSnapshot: follower nextIndex: 0, leader snapshotIndex: 5, leader lastIndex: 5, leader log size: 0 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Capturing Snapshot : lastLogEntry is null. Using snapshot values lastAppliedIndex 5 and lastAppliedTerm 1 instead. 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=5, lastAppliedTerm=1, lastIndex=5, lastTerm=1, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=false] to install on member-2-shard-cars-testInstallSnapshot 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: lastSequenceNumber prior to capture: 4 09:31:55,354 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[]}}} 09:31:55,355 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testInstallSnapshot Received follower initial sync status for member-2-shard-cars-testInstallSnapshot status sync done true 09:31:55,356 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testInstallSnapshot , received role change from Candidate to Leader 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: CaptureSnapshotReply received by actor 09:31:55,356 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=member-1-shard-cars-testInstallSnapshot, leaderPayloadVersion=12] 09:31:55,356 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:55,356 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:55,356 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received role changed for member-1-shard-cars-testInstallSnapshot from Candidate to Leader 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Persisting of snapshot done: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@440cfb43, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 09:31:55,356 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testInstallSnapshot: All Shards are ready - data store testInstallSnapshot is ready 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 5 and term: 1 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): sendInstallSnapshot 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): next snapshot chunk size for follower member-2-shard-cars-testInstallSnapshot: 376 09:31:55,356 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): InstallSnapshot sent to follower Vector(user, shardmanager-testInstallSnapshot, member-2-shard-cars-testInstallSnapshot), Chunk: 1/1 09:31:55,357 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 4 09:31:55,357 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 4 09:31:55,357 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: SnapshotComplete received 09:31:55,357 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a snapshot response 09:31:55,357 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: handled a journal response 09:31:55,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot: InstallSnapshot [term=5, leaderId=member-1-shard-cars-testInstallSnapshot, lastIncludedIndex=5, lastIncludedTerm=1, datasize=376, Chunk=1/1, lastChunkHashCode=OptionalInt[-1], serverConfig=null] 09:31:55,358 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - addChunk: chunkIndex=1, lastChunkIndex=0, collectedChunks.size=0, lastChunkHashCode=-1 09:31:55,358 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): Snapshot installed from leader: member-1-shard-cars-testInstallSnapshot 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot on follower: Snapshot [lastIndex=5, lastTerm=1, lastAppliedIndex=5, lastAppliedTerm=1, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@5ce0cb4d, electionTerm=5, electionVotedFor=member-1-shard-cars-testInstallSnapshot, ServerConfigPayload=null] 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - lastSequenceNumber prior to persisting applied snapshot: 2 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: SaveSnapshotSuccess received for snapshot, sequenceNr: 2 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Snapshot success - sequence number: 2 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Moving last applied index from -1 to 5 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Set currentTerm=5, votedFor=member-1-shard-cars-testInstallSnapshot 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Applying snapshot 09:31:55,359 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-cars-testInstallSnapshot: applying snapshot FrontendShardDataTreeSnapshotMetadata{clients=[]} over clients {} 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testInstallSnapshot: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1581247460] with name member-2-shard-cars-testInstallSnapshot-DTCL-publisher 09:31:55,360 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1581247460] 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testInstallSnapshot: state snapshot applied in 232.3 μs 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testInstallSnapshot: Done applying snapshot 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot (Follower): handleInstallSnapshot returning: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 09:31:55,360 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testInstallSnapshot: publishChanges: rootPath: / 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: SnapshotComplete received 09:31:55,360 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot-DTCL-publisher#-1581247460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,360 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testInstallSnapshot: Elapsed time for generation of change events for member-2-shard-cars-testInstallSnapshot-DTCL-publisher: 78.46 μs 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a journal response 09:31:55,360 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: handled a snapshot response 09:31:55,360 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): handleInstallSnapshotReply: InstallSnapshotReply [term=5, followerId=member-2-shard-cars-testInstallSnapshot, chunkIndex=1, success=true] 09:31:55,360 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Snapshot successfully installed on follower member-2-shard-cars-testInstallSnapshot (last chunk 1) - matchIndex set to 5, nextIndex set to 6 09:31:55,372 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2f6ca47d: creating a new snapshot member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 09:31:55,372 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 09:31:55,372 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:31:55,372 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:31:55,372 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] 09:31:55,372 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:55,372 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4423fd94] 09:31:55,372 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=0} shard 1 09:31:55,372 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-0 09:31:55,372 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@4423fd94] 09:31:55,372 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:31:55,372 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1059823443], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@5f198483 09:31:55,372 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:31:55,373 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@28b5fcfd, attempting to connect 09:31:55,373 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} is not yet known 09:31:55,373 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, nanosAgo=7601, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 09:31:55,373 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4ee3c918 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cd968b7 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cd968b7 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 for transaction member-1-datastore-testInstallSnapshot-fe-0-txn-0-1 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1059823443], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2f6ca47d finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@63d36714 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@72440e5d superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@52f4c72, splicing queue 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@72440e5d queue spliced 0 messages 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 294.1 μs 09:31:55,373 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0} 09:31:55,373 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testInstallSnapshot, generation=0} 09:31:55,374 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-2-shard-cars-testInstallSnapshot 09:31:55,374 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 09:31:55,374 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: onShutDown 09:31:55,374 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testInstallSnapshot 09:31:55,374 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testInstallSnapshot (Follower): Closing 09:31:55,374 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 09:31:55,374 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 09:31:55,374 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 09:31:55,384 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 09:31:55,384 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0} 09:31:55,384 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testInstallSnapshot: shutting down 09:31:55,384 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1670712933] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1670712933] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,384 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-2-frontend-datastore-testInstallSnapshot: Closing 09:31:55,385 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1670712933] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#1670712933] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,385 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1877135463] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1877135463] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Issuing gracefulStop to shard member-1-shard-cars-testInstallSnapshot 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testInstallSnapshot - waiting on 1 shards 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: onShutDown 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Initiating leader transfer 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37765c20 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testInstallSnapshot: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@37765c20 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Attempting to transfer leadership 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testInstallSnapshot: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 5 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testInstallSnapshot: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testInstallSnapshot, leaderId=null, leaderPayloadVersion=12] 09:31:55,385 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@34687261 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4c7b583f, splicing queue 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@34687261 queue spliced 0 messages 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] ClientActorBehavior - member-1-frontend-datastore-testInstallSnapshot: refreshing backend for shard 1 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:31:55,386 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] 09:31:55,386 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:31:55,387 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testInstallSnapshot, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:31:55,387 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,388 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-2-shard-cars-testInstallSnapshot] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,586 AM [cluster-test-shard-dispatcher-17] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testInstallSnapshot: Failed to transfer leadership in 200.9 ms 09:31:55,586 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: In unpauseLeader 09:31:55,586 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: leader transfer failed - sending PoisonPill 09:31:55,586 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot (Leader): Leadership transfer expired 09:31:55,588 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-cars-testInstallSnapshot 09:31:55,588 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testInstallSnapshot (Leader)_slicer-id-795: Closing 09:31:55,588 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testInstallSnapshot: Aborting 0 pending queued transactions 09:31:55,588 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testInstallSnapshot: All shards shutdown - sending PoisonPill to self 09:31:55,588 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testInstallSnapshot 09:31:55,588 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 09:31:55,589 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testInstallSnapshot: Closing 09:31:55,589 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot#12954106] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:55,590 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:55,590 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:55,594 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:55,595 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:55,595 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:55,595 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:55,597 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:55,611 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:55,611 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:56,610 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8965649875919904103] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:56,629 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [793452295753815687] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:56,632 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:56,632 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:56,634 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:56.636] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:56,642 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:56,642 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:56,644 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:31:56.645] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:56,658 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:56,665 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-3966877102541056555] 09:31:56,665 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:31:56,667 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:31:56,667 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:31:56,667 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:56,667 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:56,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:31:56,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:31:56,679 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:31:56,684 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:31:56,691 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [6187919174845112865] 09:31:56,691 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:31:56,692 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:56,692 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:31:56,699 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:56,699 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:56,708 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:31:56,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5318100237591145323] 09:31:56,721 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:31:56,736 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:56,737 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:56,739 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:31:56,740 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:31:56,740 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:31:56,740 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:31:56,740 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:31:56,746 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1358604070] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,746 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:56,768 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:56,768 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,768 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:56,769 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,769 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:31:56,769 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:31:56,769 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@36c219de to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@56ede691 09:31:56,769 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@54b85bea 09:31:56,769 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037516769, None)) 09:31:56,770 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037516768,0,0)) 09:31:56,770 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@56ede691 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6b437652 09:31:56,770 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@42b38870 09:31:56,772 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:31:56,772 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:56,773 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:56,773 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,774 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:56,776 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:31:56,776 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:31:56,777 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:31:56,777 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:31:56,779 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,779 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#784817661] 09:31:56,779 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1281174518] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,782 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:31:56,782 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:56,783 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:56,783 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:56,784 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 09:31:56,784 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 09:31:56,786 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:31:56,789 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:56,790 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,790 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:31:56,791 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:56,791 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:31:56,800 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-1:snapshot-read#784817661] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:56,801 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:31:56,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,801 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:31:56,801 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@49e47d2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20c58468 09:31:56,801 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@34801ce9 09:31:56,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:56,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,802 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037516801, None)) 09:31:56,802 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037516800,0,0)) 09:31:56,802 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@20c58468 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@78668e24 09:31:56,802 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@10e3c537 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,803 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1484915951] 09:31:56,804 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:31:56,804 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:31:56,805 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1484915951] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,805 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1455542835 created and ready for shard:member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,805 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-100061828] 09:31:56,805 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1455542835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,805 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-2:snapshot-read#-100061828] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:56,805 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:31:56,805 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:31:56,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Initializing shard [cars] 09:31:56,805 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is initialized 09:31:56,805 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:56,806 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:31:56,806 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 09:31:56,806 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,806 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 09:31:56,806 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 09:31:56,807 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#-1281174518 created and ready for shard:member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:56,807 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from null to Follower 09:31:56,807 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , registered listener akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader 09:31:56,807 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from null to Follower 09:31:56,853 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1358604070] 09:31:56,853 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:31:56,853 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Received ElectionTimeout - switching to Candidate 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 1 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader is no longer the leader 09:31:57,26 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Follower to Candidate 09:31:57,26 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Follower to Candidate 09:31:57,285 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:31:57,285 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,285 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 2 09:31:57,402 AM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:31:57,536 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:31:57,536 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,536 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 3 09:31:57,686 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:31:57,686 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:31:57,686 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:57,686 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,686 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,687 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:57,687 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,687 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,687 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,687 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,732 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:57,732 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,732 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,732 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:57,732 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,733 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,733 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,746 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:31:57,795 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Received ElectionTimeout 09:31:57,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,796 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Starting new election term 4 09:31:57,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,797 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:31:57,797 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [] 09:31:57,797 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:31:57,798 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:31:57,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=null 09:31:57,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:31:57,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,798 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:31:57,798 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Election: Leader has following peers: [member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:31:57,799 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: became leader with frontend state for [] 09:31:57,800 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader , received role change from Candidate to Leader 09:31:57,800 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=12] 09:31:57,800 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:57,800 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:57,800 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received role changed for member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader from Candidate to Leader 09:31:57,800 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Closing 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: getLeaderAddress leaderId = member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done false 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderPayloadVersion=12] 09:31:57,800 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:31:57,800 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All Shards are ready - data store testSemiReachableCandidateNotDroppingLeader is ready 09:31:57,801 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader updated: matchIndex: -1, nextIndex: 0 09:31:57,801 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:31:57,810 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:57,812 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:31:57,814 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSemiReachableCandidateNotDroppingLeader 09:31:57,818 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:57,821 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#756245425] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Creating local shard: member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,824 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: no initial generation override, starting from 0 09:31:57,824 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:31:57,824 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@45f8e216 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54c191cc 09:31:57,824 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@50fe0f8c 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:31:57,824 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,825 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, 0, 1714037517824, None)) 09:31:57,825 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037517823,0,0)) 09:31:57,825 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@796a4545 09:31:57,825 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@54c191cc to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@22e013fa 09:31:57,825 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} started 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1678329964] 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1678329964] 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader to akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: updated peer member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader on shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1678329964] 09:31:57,826 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader, persistent : true 09:31:57,826 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1678329964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:57,826 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1623640940 created and ready for shard:member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,826 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader-notifier#1623640940] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:31:57,826 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader/shard-member-3:snapshot-read#-1356970095] 09:31:57,827 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:31:57,827 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader with journal batch size 1 09:31:57,827 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:31:57,827 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:31:57,827 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:31:57,827 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:31:57,827 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:31:57,835 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:57,835 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:57,835 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:57,835 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:31:57,837 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:31:57,843 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#-1358604070] 09:31:57,846 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:31:57,846 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:31:57,846 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:31:57,917 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader Received follower initial sync status for member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader status sync done true 09:31:58,706 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:31:58,843 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [5318100237591145323] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:31:58,845 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2557] having UID [5318100237591145323] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:00,78 AM [cluster-test-shard-dispatcher-11] [WARN] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 213, lastApplied : -1, commitIndex : -1 09:32:03,805 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 09:32:03,806 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:03,806 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:03,809 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:03,833 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking node as UNREACHABLE [Member(akka://cluster-test@127.0.0.1:2557, Up)]. 09:32:03,986 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] 09:32:03,986 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: findLocalShard : cars 09:32:03,986 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1484915951] 09:32:03,987 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=5, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Cluster state: [Member(akka://cluster-test@127.0.0.1:2557, Up)] 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Unreachable member: Member(akka://cluster-test@127.0.0.1:2557, Up), matches candidateId in: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1], not updating term 09:32:03,987 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): In requestVote: RequestVote [term=7, candidateId=member-3-shard-cars, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, lastIndex: -1, lastTerm: -1 09:32:03,987 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): requestVote returning: RequestVoteReply [term=4, voteGranted=false] 09:32:03,987 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1484915951] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,987 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader#-1484915951] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,987 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] 09:32:03,988 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/deadLetters] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,988 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemoteDeadLetterActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVoteReply] from Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/deadLetters] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/deadLetters] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,988 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:32:03,988 AM [main] [WARN] ActorUtils - ShardManager for testSemiReachableCandidateNotDroppingLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testSemiReachableCandidateNotDroppingLeader(DistributedDataStoreRemotingIntegrationTest.java:1322) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:32:03,988 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:32:03,988 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:32:03,991 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:32:03,992 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 09:32:03,992 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 09:32:03,992 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:32:03,992 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader (Follower): Closing 09:32:03,992 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:32:03,992 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 09:32:03,992 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:32:03,993 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:32:03,993 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader, generation=0} 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Issuing gracefulStop to shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader - waiting on 1 shards 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: onShutDown 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Initiating leader transfer 09:32:03,993 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: shutting down 09:32:03,993 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@47e8d517 09:32:03,993 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@47e8d517 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Attempting to transfer leadership 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader, leaderId=null, leaderPayloadVersion=12] 09:32:03,993 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader#763680460] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:03,993 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:03,993 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#112918225] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#112918225] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:03,995 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader#2055190062] to Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSemiReachableCandidateNotDroppingLeader/member-2-shard-cars-testSemiReachableCandidateNotDroppingLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:04,285 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Failed to transfer leadership in 292.3 ms 09:32:04,286 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: In unpauseLeader 09:32:04,286 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: leader transfer failed - sending PoisonPill 09:32:04,286 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader): Leadership transfer expired 09:32:04,286 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader 09:32:04,286 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader (Leader)_slicer-id-800: Closing 09:32:04,287 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-1-shard-cars-testSemiReachableCandidateNotDroppingLeader: Aborting 0 pending queued transactions 09:32:04,287 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testSemiReachableCandidateNotDroppingLeader: All shards shutdown - sending PoisonPill to self 09:32:04,287 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Stopping ShardManager shard-manager-testSemiReachableCandidateNotDroppingLeader 09:32:04,288 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:32:04,288 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSemiReachableCandidateNotDroppingLeader: Closing 09:32:04,288 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:04,288 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:05,302 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-3966877102541056555] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:05,307 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:05,307 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:05,308 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:05.309] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:05,327 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:05,327 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:05,330 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:05.332] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:05,351 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:05,358 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [9217300686316402512] 09:32:05,358 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:05,372 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:05,372 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:05,372 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:05,372 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:05,372 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:05,373 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:05,373 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:05,376 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:05,387 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3657783492989761887] 09:32:05,387 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:05,389 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:05,389 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:05,399 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:05,399 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:05,405 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:05,424 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7103303638626108062] 09:32:05,425 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:05,426 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:05,426 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:05,434 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:05,434 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:05,436 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:05,438 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:05,439 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 09:32:05,442 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1258362416] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,444 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:05,451 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:05,451 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,451 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:05,451 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,453 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 09:32:05,453 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:32:05,453 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@12b6dd30 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c2ee636 09:32:05,453 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2811776b 09:32:05,455 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1714037525453, None)) 09:32:05,455 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037525452,0,0)) 09:32:05,455 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2c2ee636 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@ed21b2b 09:32:05,455 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@697e0d2 09:32:05,455 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 09:32:05,456 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:05,456 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,456 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:05,456 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,456 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,456 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481] 09:32:05,457 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 09:32:05,457 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 09:32:05,457 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,457 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-770189769] 09:32:05,457 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1371271673 created and ready for shard:member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,458 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1371271673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,458 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-1:snapshot-read#-770189769] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,458 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:05,460 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadyLocalTransactionForwardedToLeader 09:32:05,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:05,463 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:05,463 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 09:32:05,463 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 09:32:05,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:05,463 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:32:05,463 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 09:32:05,463 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 09:32:05,463 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 09:32:05,467 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:05,471 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:05,471 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,471 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:05,474 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:05,475 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: no initial generation override, starting from 0 09:32:05,475 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:32:05,475 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6c329eb8 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46b24203 09:32:05,475 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1e083d2b 09:32:05,476 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, 0, 1714037525476, None)) 09:32:05,476 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037525475,0,0)) 09:32:05,476 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46b24203 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@e44718b 09:32:05,476 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:05,476 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} started 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@28c86e69 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:05,478 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,479 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,479 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: updated peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader on shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] 09:32:05,479 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, persistent : true 09:32:05,479 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with journal batch size 1 09:32:05,480 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,480 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1088886479] 09:32:05,480 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1327380355 created and ready for shard:member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,480 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/shard-member-2:snapshot-read#1088886479] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,480 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-notifier#1327380355] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:05,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Initializing shard [cars] 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:05,481 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is initialized 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:32:05,481 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from null to Follower 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,481 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader 09:32:05,481 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader from null to Follower 09:32:05,536 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1258362416] 09:32:05,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:32:05,536 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481] 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 1 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader is no longer the leader 09:32:05,688 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Follower to Candidate 09:32:05,688 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Follower to Candidate 09:32:05,997 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:32:05,997 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:05,998 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 2 09:32:06,41 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:06,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:32:06,307 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,307 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 3 09:32:06,378 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:06,378 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,379 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,379 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-3-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,390 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,440 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:06,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:32:06,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,558 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Starting new election term 4 09:32:06,558 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Cluster state: [] 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,559 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: became leader with frontend state for [] 09:32:06,560 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader , received role change from Candidate to Leader 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=12] 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader from Candidate to Leader 09:32:06,560 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Closing 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderPayloadVersion=12] 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All Shards are ready - data store testReadyLocalTransactionForwardedToLeader is ready 09:32:06,563 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done false 09:32:06,563 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 09:32:06,564 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:06,642 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] 09:32:06,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:32:06,643 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] 09:32:06,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,643 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] 09:32:06,643 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: findLocalShard : cars 09:32:06,643 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] 09:32:06,654 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,654 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,654 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,656 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-97-0 of 1 operations, client version 12 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-97-0 ready 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 validated 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-97-0: success 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 candidate ready 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: canCommit - candidate rootPath: / 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: canCommit - messages: [] 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: preCommit - successfulFromPrevious: [] 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 prepared 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-97-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,657 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 submitted to persistence 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 1 09:32:06,658 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:06,658 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:32:06,658 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:32:06,659 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader status sync done true 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 0 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=2765598} is already disabled 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:32:06,659 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 0 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 0 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:32:06,660 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 1 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-97-0: commit - successfulFromPrevious: [] 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 changing state from COMMIT_PENDING to COMMITTED 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-97-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-650#1623339663] 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#961795588] with name member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 09:32:06,661 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#961795588] 09:32:06,661 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:32:06,661 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#961795588] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,661 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 114.2 μs 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:06,661 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Commit index set to 1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Setting last applied to 1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:32:06,664 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@34847951: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-97-0 09:32:06,664 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 09:32:06,664 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:06,664 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:06,664 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1258362416] 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1699389964] with name member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher 09:32:06,664 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 09:32:06,664 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:32:06,665 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-0 09:32:06,665 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:32:06,665 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:06,665 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#182860341], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@34b1e5f 09:32:06,665 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1699389964] 09:32:06,665 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:06,665 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:32:06,665 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 52.30 μs 09:32:06,665 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher#1699389964] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,665 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:06,665 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33503f11] 09:32:06,668 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33503f11] 09:32:06,668 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@674f0da0, attempting to connect 09:32:06,668 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} is not yet known 09:32:06,668 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, nanosAgo=7251, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:32:06,668 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:06,668 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:06,668 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:06,668 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7e8821c9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25546fb7 09:32:06,668 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25546fb7 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#182860341], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@34847951 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5d60650f 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@74985387 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@72b7a374, splicing queue 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@74985387 queue spliced 0 messages 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 464.3 μs 09:32:06,669 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 09:32:06,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: handleReadyLocalTransaction for member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testReadyLocalTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,669 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Forwarding ReadyLocalTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader)] 09:32:06,670 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,670 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-98-0 of 1 operations, client version 12 09:32:06,670 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,670 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-98-0 ready 09:32:06,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 canCommit 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481)] resolved 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: successfully resolved all cohort actors 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 finishCanCommit 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-98-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481)] 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481)] 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 validated 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-98-0: success 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:32:06,671 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit returning result: true 09:32:06,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 commit 09:32:06,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0 finish commit 09:32:06,671 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-98-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481)] 09:32:06,671 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481)] 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 candidate ready 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit - candidate rootPath: / 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: canCommit - messages: [] 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: preCommit - successfulFromPrevious: [] 09:32:06,671 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 prepared 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-98-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 submitted to persistence 09:32:06,672 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Log size is now 2 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadyLocalTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:06,673 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadyLocalTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Setting last applied to 2 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Moving last applied index from 1 to 2 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-98-0 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-98-0: commit - successfulFromPrevious: [] 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 changing state from COMMIT_PENDING to COMMITTED 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-98-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testReadyLocalTransactionForwardedToLeader_member-1-shard-cars-testReadyLocalTransactionForwardedToLeader%231224044481$a] 09:32:06,680 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-98-0: commit succeeded 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:32:06,680 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 09:32:06,680 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: publishChanges: rootPath: / 09:32:06,680 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testReadyLocalTransactionForwardedToLeader-DTCL-publisher: 70.98 μs 09:32:06,680 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@34847951: creating a new snapshot member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 09:32:06,680 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-0 09:32:06,681 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:32:06,681 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,681 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 09:32:06,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 09:32:06,681 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader (Follower): Closing 09:32:06,681 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:32:06,681 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader#1442223247] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,681 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:32:06,681 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:06,682 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:32:06,682 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0} 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader - waiting on 1 shards 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#414675352] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#414675352] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: onShutDown 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Initiating leader transfer 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#-2073913643] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,682 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=12] 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f0307ce 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@4f0307ce 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Attempting to transfer leadership 09:32:06,682 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:06,682 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadyLocalTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:32:06,682 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6afd911f superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7a235eb7, splicing queue 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6afd911f queue spliced 0 messages 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: refreshing backend for shard 1 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:06,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader#1258362416] 09:32:06,683 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:06,683 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testReadyLocalTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:32:06,684 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481] to Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-2-shard-cars-testReadyLocalTransactionForwardedToLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:06,918 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Failed to transfer leadership in 235.7 ms 09:32:06,918 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: In unpauseLeader 09:32:06,918 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: leader transfer failed - sending PoisonPill 09:32:06,918 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader): Leadership transfer expired 09:32:06,919 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadyLocalTransactionForwardedToLeader 09:32:06,919 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader (Leader)_slicer-id-807: Closing 09:32:06,919 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadyLocalTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:32:06,919 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testReadyLocalTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:32:06,919 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadyLocalTransactionForwardedToLeader 09:32:06,919 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:32:06,919 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: shutting down 09:32:06,919 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: Closing 09:32:06,926 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:06,926 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:06,930 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:06,930 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:06,931 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:06,931 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:06,940 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:06,940 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:07,939 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [9217300686316402512] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:07,953 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:07,954 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:07.957] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:07,959 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3657783492989761887] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:07,966 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:07,966 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:32:07.969] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:07,968 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:07,985 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:07,991 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-9070753762900960874] 09:32:07,992 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:07,993 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:07,993 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:07,995 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:07,995 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:08,12 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25546fb7 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 09:32:08,12 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#182860341]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@355b4878 09:32:08,12 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 09:32:08,12 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:08,12 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:08,12 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:08,13 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7e8821c9 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:08,13 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#182860341], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@29ea9c66 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1} 09:32:08,13 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@25546fb7 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testReadyLocalTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#182860341]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@357efeb1 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testReadyLocalTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testReadyLocalTransactionForwardedToLeader/member-1-shard-cars-testReadyLocalTransactionForwardedToLeader#1224044481], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:08,13 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4ee3c918 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1059823443], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@3bd94836 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1} 09:32:08,13 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3cd968b7 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testInstallSnapshot-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1059823443]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@54aedfbd 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 09:32:08,13 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testInstallSnapshot: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testInstallSnapshot, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testInstallSnapshot/member-1-shard-cars-testInstallSnapshot#2071673653], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:08,16 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:08,27 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6756616128729534710] 09:32:08,28 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:08,29 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:08,29 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:08,40 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:08,40 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:08,52 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:08,63 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5630198426216373829] 09:32:08,63 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:08,65 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:08,65 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:08,67 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:08,67 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:08,78 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:08,79 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:08,81 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 09:32:08,101 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1168544236] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,102 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:32:08,103 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:32:08,103 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:32:08,103 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2a0799c1 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e68f7e5 09:32:08,104 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@35f230da 09:32:08,104 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1714037528103, None)) 09:32:08,105 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037528102,0,0)) 09:32:08,105 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@e68f7e5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5cf2fcba 09:32:08,105 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 09:32:08,105 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:08,107 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:08,109 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithSingleShard 09:32:08,114 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:32:08,114 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,119 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:32:08,119 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:32:08,119 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@cc4189e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7845ef3b 09:32:08,119 AM [cluster-test-shard-dispatcher-16] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 09:32:08,119 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,125 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@54e90845 09:32:08,126 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, 0, 1714037528119, None)) 09:32:08,129 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037528118,0,0)) 09:32:08,129 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7845ef3b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4b18e21 09:32:08,129 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} started 09:32:08,131 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@11c98947 09:32:08,131 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:08,131 AM [cluster-test-shard-dispatcher-16] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,134 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithSingleShard 09:32:08,134 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,135 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:08,135 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,135 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7368c6a7 09:32:08,135 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:08,135 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,146 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 09:32:08,148 AM [cluster-test-shard-dispatcher-16] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 09:32:08,150 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:08,153 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:08,153 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard#409613365] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,153 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-2144822510 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,154 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-2144822510] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,154 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-838941259] 09:32:08,155 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:08,155 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,155 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,155 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-838941259] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,157 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:08,157 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:08,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 09:32:08,157 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is initialized 09:32:08,159 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:08,159 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:32:08,159 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 09:32:08,159 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,159 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 09:32:08,159 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 09:32:08,164 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,164 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testReadWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard on shard member-1-shard-cars-testReadWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103] 09:32:08,165 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithSingleShard, persistent : true 09:32:08,166 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithSingleShard with journal batch size 1 09:32:08,166 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,166 AM [cluster-test-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-501060939] 09:32:08,166 AM [cluster-test-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-384638830] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,166 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-notifier#-384638830 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,167 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/shard-member-1:snapshot-read#-501060939] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:08,167 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:08,167 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:08,167 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:08,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Initializing shard [cars] 09:32:08,167 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:32:08,167 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is initialized 09:32:08,167 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from null to Follower 09:32:08,167 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard 09:32:08,167 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from null to Follower 09:32:08,168 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:08,170 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:08,180 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1168544236] 09:32:08,180 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: findLocalShard : cars 09:32:08,180 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103] 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:08,434 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithSingleShard is no longer the leader 09:32:08,434 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:32:08,435 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Follower to Candidate 09:32:08,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:08,723 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:08,723 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 2 09:32:08,763 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:09,13 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:09,13 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,13 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 3 09:32:09,14 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:09,14 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,15 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithSingleShard/member-3-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,84 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:09,293 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:09,293 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,293 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Starting new election term 4 09:32:09,293 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,295 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:09,295 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Cluster state: [] 09:32:09,295 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:09,295 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:09,295 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:32:09,296 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:09,296 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,296 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithSingleShard] 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: became leader with frontend state for [] 09:32:09,297 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testReadWriteTransactionWithSingleShard from Candidate to Leader 09:32:09,297 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 09:32:09,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Closing 09:32:09,298 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,298 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:09,298 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:09,298 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:09,299 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All Shards are ready - data store testReadWriteTransactionWithSingleShard is ready 09:32:09,299 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done false 09:32:09,300 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:32:09,300 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:09,337 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e2593f1: creating a new transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:09,337 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:09,337 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:09,337 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:09,337 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] 09:32:09,337 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 09:32:09,337 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:09,337 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:09,337 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:09,337 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:32:09,337 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=12] 09:32:09,337 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@3800c94 09:32:09,337 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:09,337 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@6928631f 09:32:09,337 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard, primaryVersion=12] 09:32:09,338 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@56e2caf7, attempting to connect 09:32:09,339 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} is not yet known 09:32:09,339 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, nanosAgo=8950, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:32:09,340 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:09,340 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:09,340 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:09,340 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5493f7c2 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b2b65f6 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b2b65f6 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2299/0x000000080170d058@2a5591af 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3}, enqueuedTicks=735631226008} 09:32:09,341 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3}} 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message size: 468 bytes, total slices to send: 1 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message does not need to be sliced - sending original message 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@566e10dc 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=735631226008} 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e2593f1 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@33f6931d 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6eaa918f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6b6ec26d, splicing queue 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6eaa918f queue spliced 0 messages 09:32:09,342 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.340 ms 09:32:09,343 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,344 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 09:32:09,344 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: Message size: 372 bytes, total slices to send: 1 09:32:09,344 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: Message does not need to be sliced - sending original message 09:32:09,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:09,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3}, enqueuedTicks=735631226008, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:09,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=3} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 09:32:09,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 09:32:09,345 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=735631226008, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 09:32:09,346 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1} 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@188f7264 09:32:09,346 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 2 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1324/0x000000080135cf70@5e43fecd 09:32:09,346 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@5ed1b917 09:32:09,346 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:09,346 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1}, enqueuedTicks=735639687032} 09:32:09,346 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1}} 09:32:09,346 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message size: 458 bytes, total slices to send: 1 09:32:09,346 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message does not need to be sliced - sending original message 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 3 09:32:09,346 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1321/0x000000080135c8d8@458b22fa 09:32:09,346 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=735639923946} 09:32:09,351 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 09:32:09,351 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1}, enqueuedTicks=735639687032, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2}} 09:32:09,351 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=2} 09:32:09,355 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 09:32:09,355 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=735639923946, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true}} 09:32:09,355 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=3, exists=true} 09:32:09,355 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e2593f1 readied transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:09,355 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} allocated sequence 4 09:32:09,355 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@2ac1181c 09:32:09,355 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:09,355 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=SIMPLE}, enqueuedTicks=735648783478} 09:32:09,355 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=SIMPLE}} 09:32:09,355 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message size: 290 bytes, total slices to send: 1 09:32:09,355 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message does not need to be sliced - sending original message 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: readying transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: transitioned member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 09:32:09,356 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 250 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 09:32:09,357 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}]]] 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=250} 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Log size is now 1 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:09,358 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 09:32:09,358 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=250}] 09:32:09,359 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithSingleShard status sync done true 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Setting last applied to 0 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@f513be0}, purged=MutableUnsignedLongSet{size=0}} 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-858071121] with name member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 09:32:09,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:09,360 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:32:09,360 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-858071121] 09:32:09,360 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:09,360 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-858071121] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,360 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 58.98 μs 09:32:09,360 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=SIMPLE}, enqueuedTicks=735648783478, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=4}} 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Commit index set to 0 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} directCommit completed 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED}: initiating purge 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 5 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Setting last applied to 0 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1661517728]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4d20d72c 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=250} 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:09,361 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#-1661517728]}, enqueuedTicks=735654347252} 09:32:09,361 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@5e2593f1: creating a new snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:09,361 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:09,361 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:09,361 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@718dde2 09:32:09,361 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=735654833361} 09:32:09,361 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,362 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1a7f25ac}, purged=MutableUnsignedLongSet{size=0}} 09:32:09,362 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:09,362 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,362 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1109666534] with name member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher 09:32:09,362 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1109666534] 09:32:09,362 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:09,362 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher: 49.69 μs 09:32:09,362 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard-DTCL-publisher#-1109666534] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,363 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:32:09,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: purging transaction member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:09,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 09:32:09,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 09:32:09,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:09,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1 09:32:09,364 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:09,364 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: Message size: 395 bytes, total slices to send: 1 09:32:09,364 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard_slicer-id-811: Message does not need to be sliced - sending original message 09:32:09,364 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}] 09:32:09,365 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:09,365 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=735654833361, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:09,365 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 09:32:09,365 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:32:09,365 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,365 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 09:32:09,365 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 09:32:09,366 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithSingleShard (Follower): Closing 09:32:09,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:09,366 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:32:09,366 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:32:09,366 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:32:09,366 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithSingleShard, generation=0} 09:32:09,367 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: shutting down 09:32:09,367 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:32:09,367 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,367 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,367 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithSingleShard - waiting on 1 shards 09:32:09,367 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard#-1215872117] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,367 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1661517728] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1661517728] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: onShutDown 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Initiating leader transfer 09:32:09,367 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:32:09,367 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:09,367 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e23da2f 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2e23da2f 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Attempting to transfer leadership 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,367 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 09:32:09,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,369 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithSingleShard/member-1-shard-cars-testReadWriteTransactionWithSingleShard#1036484103] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithSingleShard/member-2-shard-cars-testReadWriteTransactionWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:09,533 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,533 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-0-1, size=69}]]] 09:32:09,653 AM [cluster-test-shard-dispatcher-25] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Failed to transfer leadership in 285.6 ms 09:32:09,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: In unpauseLeader 09:32:09,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: leader transfer failed - sending PoisonPill 09:32:09,653 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader): Leadership transfer expired 09:32:09,654 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithSingleShard 09:32:09,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithSingleShard (Leader)_slicer-id-812: Closing 09:32:09,654 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:09,654 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:32:09,654 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithSingleShard 09:32:09,654 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:32:09,655 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithSingleShard: Closing 09:32:09,655 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:09,655 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:32:09.660] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1040)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:09.660] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1040)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:09.661] [cluster-test-akka.remote.default-remote-dispatcher-5] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1040)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:09.661] [cluster-test-akka.remote.default-remote-dispatcher-8] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1040)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:09,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:09,675 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:10,673 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-9070753762900960874] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:10,693 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6756616128729534710] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:10,696 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:10,696 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:10.698] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:10,705 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:10,705 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:10,707 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:10.708] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:10,720 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:32:10,725 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [1891161438011720342] 09:32:10,725 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:10,726 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:10,726 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:10,726 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:10,726 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:10,736 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:10,736 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:10,736 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:10,737 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:10,742 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [1339944379520911183] 09:32:10,742 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:10,744 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:10,745 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:10,754 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:10,754 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:10,768 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:10,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-7896834452581270924] 09:32:10,773 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:10,776 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:10,777 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:10,783 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:10,783 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:10,784 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:10,786 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:10,787 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 09:32:10,788 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:10,788 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:10,793 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,793 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6a3561af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@563d7d76 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@133ff9d5 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1714037530797, None)) 09:32:10,797 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037530796,0,0)) 09:32:10,798 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@563d7d76 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2266aa08 09:32:10,798 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@68082dfb 09:32:10,798 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 09:32:10,798 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:10,800 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:10,801 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-1-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744] 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Shard created : member-1-shard-cars-testSnapshotOnRootOverwrite, persistent : true 09:32:10,802 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-1-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125] 09:32:10,802 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-default-testSnapshotOnRootOverwrite, persistent : true 09:32:10,802 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 09:32:10,802 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,804 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSnapshotOnRootOverwrite 09:32:10,804 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-1564888581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,806 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:10,806 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:10,806 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:10,806 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:32:10,806 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 09:32:10,806 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSnapshotOnRootOverwrite is initialized 09:32:10,807 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-1789758774 created and ready for shard:member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,807 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 09:32:10,807 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:32:10,807 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from null to Follower 09:32:10,807 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-notifier#-1789758774] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,812 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 09:32:10,812 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-115572741] 09:32:10,812 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-115572741] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,812 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-763447787] 09:32:10,812 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/shard-member-1:snapshot-read#-763447787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 09:32:10,813 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-default-testSnapshotOnRootOverwrite is initialized 09:32:10,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-notifier#-1564888581 created and ready for shard:member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 09:32:10,813 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:32:10,813 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from null to Follower 09:32:10,814 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#36474359] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSnapshotOnRootOverwrite 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:10,816 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: no initial generation override, starting from 0 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Creating local shard: member-2-shard-default-testSnapshotOnRootOverwrite 09:32:10,816 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:10,816 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6bc6a67 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46cb90ae 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,816 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,816 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@535e3eca 09:32:10,817 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSnapshotOnRootOverwrite, 0, 1714037530816, None)) 09:32:10,818 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:10,818 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:10,820 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSnapshotOnRootOverwrite: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037530815,0,0)) 09:32:10,820 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@46cb90ae to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1477beca 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@56e68634 09:32:10,821 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} started 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-cars-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite on shard member-2-shard-cars-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#2032863051] 09:32:10,821 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-default-testSnapshotOnRootOverwrite 09:32:10,822 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testSnapshotOnRootOverwrite, persistent : true 09:32:10,822 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:10,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: updated peer member-1-shard-default-testSnapshotOnRootOverwrite on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite on shard member-2-shard-default-testSnapshotOnRootOverwrite whose actor address is Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#1041510827] 09:32:10,822 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-default-testSnapshotOnRootOverwrite, persistent : true 09:32:10,822 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSnapshotOnRootOverwrite with journal batch size 1 09:32:10,822 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-default-testSnapshotOnRootOverwrite with journal batch size 1 09:32:10,822 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#473588732] 09:32:10,822 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite#2032863051] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,822 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#315664406] 09:32:10,822 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1721327078 created and ready for shard:member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite#1041510827] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#473588732] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-notifier#1721327078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#315664406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#679797039 created and ready for shard:member-2-shard-default-testSnapshotOnRootOverwrite 09:32:10,823 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-notifier#679797039] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:10,823 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:10,823 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:10,823 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:10,823 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:10,823 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [cars] 09:32:10,823 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:10,823 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:10,823 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSnapshotOnRootOverwrite is initialized 09:32:10,823 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:32:10,823 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-2-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:32:10,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Initializing shard [default] 09:32:10,824 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-default-testSnapshotOnRootOverwrite is initialized 09:32:10,824 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , received role change from null to Follower 09:32:10,824 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:10,824 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:10,824 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , received role change from null to Follower 09:32:10,824 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:32:10,824 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-default-testSnapshotOnRootOverwrite , registered listener akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite 09:32:10,824 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-default-testSnapshotOnRootOverwrite from null to Follower 09:32:10,824 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-2-shard-cars-testSnapshotOnRootOverwrite from null to Follower 09:32:10,871 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:10,871 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : cars 09:32:10,871 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744] 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Received ElectionTimeout - switching to Candidate 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Election: Candidate has following voting peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=1, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 1 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: false, hasLeader : false 09:32:11,48 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSnapshotOnRootOverwrite is no longer the leader 09:32:11,48 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onStateChanged: Closing all transaction chains because shard member-1-shard-default-testSnapshotOnRootOverwrite is no longer the leader 09:32:11,49 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 09:32:11,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Follower to Candidate 09:32:11,49 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Follower to Candidate 09:32:11,49 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Follower to Candidate 09:32:11,267 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,267 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,267 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 09:32:11,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,308 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=2, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,308 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 2 09:32:11,377 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:11,426 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:11,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,498 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,498 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 09:32:11,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=3, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,557 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 3 09:32:11,738 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:11,738 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,739 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,739 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,739 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-default-testSnapshotOnRootOverwrite set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,751 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:11,751 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite 09:32:11,751 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,751 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSnapshotOnRootOverwrite/member-3-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,752 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-default-testSnapshotOnRootOverwrite with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite to akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,757 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,757 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,757 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 09:32:11,758 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-default-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,758 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:11,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:11,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:11,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-default-testSnapshotOnRootOverwrite] 09:32:11,759 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-default-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: became leader with frontend state for [] 09:32:11,760 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-default-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:32:11,760 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-default-testSnapshotOnRootOverwrite from Candidate to Leader 09:32:11,760 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:32:11,760 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite: Closing 09:32:11,760 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-default-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite 09:32:11,760 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-default-testSnapshotOnRootOverwrite, leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:32:11,760 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:32:11,760 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done false 09:32:11,760 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:32:11,761 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 09:32:11,761 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:11,804 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:11,877 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Received ElectionTimeout 09:32:11,877 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,877 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Starting new election term 4 09:32:11,877 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Cluster state: [] 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=null 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSnapshotOnRootOverwrite, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Set currentTerm=4, votedFor=member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:11,879 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-default-testSnapshotOnRootOverwrite status sync done true 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Election: Leader has following peers: [member-2-shard-cars-testSnapshotOnRootOverwrite] 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSnapshotOnRootOverwrite: onLeadershipChange, isLeader: true, hasLeader : true 09:32:11,881 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: became leader with frontend state for [] 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite: Closing 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: getLeaderAddress leaderId = member-1-shard-cars-testSnapshotOnRootOverwrite peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:11,882 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSnapshotOnRootOverwrite , received role change from Candidate to Leader 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderPayloadVersion=12] 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received role changed for member-1-shard-cars-testSnapshotOnRootOverwrite from Candidate to Leader 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All Shards are ready - data store testSnapshotOnRootOverwrite is ready 09:32:11,882 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done false 09:32:11,883 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: -1, nextIndex: 0 09:32:11,883 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:11,976 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:11,976 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: findLocalShard : default 09:32:11,976 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125] 09:32:11,976 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@526dde11: creating a new transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,977 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 09:32:11,977 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:11,977 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:11,977 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:11,977 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:11,977 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 1 09:32:11,977 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,977 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33380d5c] 09:32:11,978 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving shard 0 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 09:32:11,978 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 09:32:11,978 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 09:32:11,978 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:11,978 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@33380d5c] 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 09:32:11,978 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} shard 0 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Found primary for default: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5a4f8f70] 09:32:11,978 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,978 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@5a4f8f70] 09:32:11,978 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@35f582f4, attempting to connect 09:32:11,978 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard default resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3d9e430, attempting to connect 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 09:32:11,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} allocated sequence 0 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=21681, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:11,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} is not yet known 09:32:11,978 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:11,978 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@526dde11 readied transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@26d45993 09:32:11,978 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, nanosAgo=10520, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:11,978 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} scheduling timeout in 30 seconds 09:32:11,979 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:11,979 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@24a2add9 09:32:11,979 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:11,979 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:11,979 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:11,979 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@568c0e97 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@14e65ce7 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@14e65ce7 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=OPEN} 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 0 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@24a2add9 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:11,980 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true}, enqueuedTicks=738272381261} 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@526dde11 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@64c4a742 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3a1675bb superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@35f4a993, splicing queue 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3a1675bb queue spliced 0 messages 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating canCommit 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 validated 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:11,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed canCommit 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.097 ms 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolved shard 0 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present} 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@554f9a7f instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@691fba82 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} previous state FLUSHED 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@691fba82 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,981 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=OPEN} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], modifications=1, protocol=THREE_PHASE} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 0 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@26d45993 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true}, enqueuedTicks=738272261789} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@526dde11 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: allocating new ready transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@25e517d9 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@a988f6f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3cd6bbb7, splicing queue 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating canCommit 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@a988f6f queue spliced 0 messages 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Validating transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 validated 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed canCommit 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} in 879.5 μs 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true}, enqueuedTicks=738272381261, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=0}} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750], coordinated=true}, enqueuedTicks=738272261789, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=0}} 09:32:11,982 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} canCommit completed 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 1 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@2203fe82 09:32:11,982 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 09:32:11,982 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 1 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@2203fe82 09:32:11,982 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:11,982 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738276184347} 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:11,982 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@261f3390 09:32:11,982 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 09:32:11,982 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating preCommit 09:32:11,982 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,983 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:11,983 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@261f3390 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 candidate ready 09:32:11,983 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - candidate rootPath: / 09:32:11,983 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738276342129} 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: canCommit - messages: [] 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: preCommit - successfulFromPrevious: [] 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 prepared 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:11,983 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed preCommit 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating preCommit 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Preparing transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 candidate ready 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: canCommit - messages: [] 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 prepared 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:11,983 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 completed preCommit 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738276184347, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=1}} 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738276342129, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=1}} 09:32:11,983 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:11,983 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 2 09:32:11,983 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@75ba6e45 09:32:11,984 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 09:32:11,984 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} allocated sequence 2 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@75ba6e45 09:32:11,984 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:11,984 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738277356436} 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@48f0699e 09:32:11,984 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 09:32:11,984 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:32:11,984 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@48f0699e 09:32:11,984 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:11,984 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738277464778} 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 initiating commit 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 113 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:32:11,984 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 submitted to persistence 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 initiating commit 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Starting commit for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 283 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 submitted to persistence 09:32:11,985 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}]]] 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=113} 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:11,989 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] 09:32:11,996 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}]]] 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Append entry to log Simple{size=283} 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSnapshotOnRootOverwrite, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] 09:32:11,997 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite Received follower initial sync status for member-2-shard-cars-testSnapshotOnRootOverwrite status sync done true 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 09:32:11,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@290003bb}, purged=MutableUnsignedLongSet{size=0}} 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0: commit - successfulFromPrevious: [] 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 changing state from COMMIT_PENDING to COMMITTED 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1154931743] with name member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738277356436, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=2}} 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} doCommit completed 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@554f9a7f completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: initiating purge 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} allocated sequence 3 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@455e34d6 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0} 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SEALED}: initiating purge 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} allocated sequence 3 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@23eb4dbf 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738291725080} 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1154931743] 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:32:11,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:32:11,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 09:32:11,999 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#-1154931743] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:11,999 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 82.22 μs 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 09:32:11,999 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@b9d2a63, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 09:32:11,999 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=113} 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSnapshotOnRootOverwrite updated: matchIndex: 0, nextIndex: 1 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-default-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4bf42b31}, purged=MutableUnsignedLongSet{size=0}} 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Setting last applied to 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-default-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Resuming commit of transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-1-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@61caf683}, purged=MutableUnsignedLongSet{size=0}} 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-default-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1647418129] with name member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 09:32:12,0 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:12,0 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1647418129] 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-default-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:32:12,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:32:12,1 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-default-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1212636849] with name member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=113}] and replicatedToAllIndex is -1 09:32:12,1 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-default-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher: 23.92 μs 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 09:32:12,1 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite/member-2-shard-default-testSnapshotOnRootOverwrite-DTCL-publisher#1647418129] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 5 09:32:12,1 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1212636849] 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738277464778, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=2}} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@568c0e97 completing transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@53de5f6c 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 100.9 μs 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1212636849] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} allocated sequence 3 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@1df3518f 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:12,2 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@526dde11: creating a new snapshot member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 09:32:12,2 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738295532725} 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@397c43d5, electionTerm=4, electionVotedFor=member-1-shard-cars-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 09:32:12,2 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-1 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:12,2 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 for transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0 09:32:12,2 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-default-testSnapshotOnRootOverwrite 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 09:32:12,2 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-default-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 09:32:12,2 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Replicate message: identifier: member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:12,3 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 5 09:32:12,3 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 5 09:32:12,3 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: SnapshotComplete received 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: CaptureSnapshotReply received by actor 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Persisting of snapshot done: Snapshot [lastIndex=0, lastTerm=4, lastAppliedIndex=0, lastAppliedTerm=4, unAppliedEntries size=0, state=org.opendaylight.controller.cluster.datastore.persisted.ShardSnapshotState@26d1b44d, electionTerm=4, electionVotedFor=member-1-shard-default-testSnapshotOnRootOverwrite, ServerConfigPayload=null] 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: user triggered or root overwrite snapshot encountered, trimming log up to last applied index 0 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Removed in-memory snapshotted entries, adjusted snaphsotIndex: 0 and term: 4 09:32:12,3 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a snapshot response 09:32:12,3 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SaveSnapshotSuccess received for snapshot, sequenceNr: 3 09:32:12,3 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Snapshot success - sequence number: 3 09:32:12,3 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: SnapshotComplete received 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Commit index set to 0 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Setting last applied to 0 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Moving last applied index from -1 to 0 09:32:12,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying state for log index 0 data Simple{size=283} 09:32:12,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: Applying foreign transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-default-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}]]] 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Number of entries to be appended = 1 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Log size is now 1 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-default-testSnapshotOnRootOverwrite, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56}] 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a snapshot response 09:32:12,4 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: handled a journal response 09:32:12,4 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: handled a journal response 09:32:12,5 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-default-testSnapshotOnRootOverwrite updated: matchIndex: 1, nextIndex: 2 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Setting last applied to 1 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Moving last applied index from 0 to 1 09:32:12,5 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, size=56} 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: replication of member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@3795d6a1 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-default-testSnapshotOnRootOverwrite: finished purging transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-default-testSnapshotOnRootOverwrite: Purged transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=1 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1068355750]}, enqueuedTicks=738291725080, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, sequence=3}} 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE}: purge completed 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@691fba82 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=DONE} 09:32:12,5 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR}: purge completed 09:32:12,5 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@554f9a7f purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-0, state=SUCCESSOR} 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}] 09:32:12,5 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testSnapshotOnRootOverwrite: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@6e448abe}, purged=MutableUnsignedLongSet{size=0}} 09:32:12,5 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-2-shard-default-testSnapshotOnRootOverwrite 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, history=0, cookie=1} 09:32:12,5 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSnapshotOnRootOverwrite: Committed transaction member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1 09:32:12,5 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: onShutDown 09:32:12,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSnapshotOnRootOverwrite: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1070731375] with name member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher 09:32:12,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1070731375] 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testSnapshotOnRootOverwrite: shard root overwritten, enqueuing snapshot 09:32:12,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSnapshotOnRootOverwrite: publishChanges: rootPath: / 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: onShutDown 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Persisting ApplyJournalEntries with index=0 09:32:12,6 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSnapshotOnRootOverwrite: Elapsed time for generation of change events for member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher: 41.00 μs 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - Take a snapshot of current state. lastReplicatedLog is SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=283}] and replicatedToAllIndex is -1 09:32:12,6 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite-DTCL-publisher#-1070731375] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Initiating snapshot capture CaptureSnapshot [lastAppliedIndex=0, lastAppliedTerm=4, lastIndex=0, lastTerm=4, installSnapshotInitiated=, replicatedToAllIndex=-1, replicatedToAllTerm=-1, unAppliedEntries size=0, mandatoryTrim=true] 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: lastSequenceNumber prior to capture: 3 09:32:12,6 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: requesting serialization of snapshot MetadataShardDataTreeSnapshot{metadata={class org.opendaylight.controller.cluster.datastore.persisted.FrontendShardDataTreeSnapshotMetadata=FrontendShardDataTreeSnapshotMetadata{clients=[FrontendClientMetadata{identifer=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, current=[FrontendHistoryMetadata{historyId=0, cookie=0, closed=false, closedTransactions={0=true}, purgedTransactions=ImmutableUnsignedLongSet{size=0}}], purged=ImmutableUnsignedLongSet{size=0}}]}}} 09:32:12,6 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-default-testSnapshotOnRootOverwrite 09:32:12,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-default-testSnapshotOnRootOverwrite (Follower): Closing 09:32:12,6 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:32:12,6 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.actors.ShardSnapshotActor$SerializeSnapshot] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#473588732] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-2-shard-cars-testSnapshotOnRootOverwrite/shard-member-2:snapshot-read#473588732] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:12,7 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:12,7 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testSnapshotOnRootOverwrite (Follower): Closing 09:32:12,8 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:32:12,8 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 09:32:12,8 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 09:32:12,8 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:32:12,8 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0} 09:32:12,8 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:12,8 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 09:32:12,8 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:32:12,8 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Issuing gracefulStop to shard member-1-shard-default-testSnapshotOnRootOverwrite 09:32:12,8 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: onShutDown 09:32:12,8 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSnapshotOnRootOverwrite - waiting on 2 shards 09:32:12,8 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Initiating leader transfer 09:32:12,8 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: onShutDown 09:32:12,8 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Initiating leader transfer 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68c6bbc4 09:32:12,9 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7821d4c2 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@68c6bbc4 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:12,9 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-default-testSnapshotOnRootOverwrite: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7821d4c2 09:32:12,9 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Attempting to transfer leadership 09:32:12,9 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Checking sendAppendEntries for follower member-2-shard-default-testSnapshotOnRootOverwrite: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=12] 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for default 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderId=null, leaderPayloadVersion=12] 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 0 is now stale 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@57c360d1, splicing queue 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@31ba70c6 queue spliced 0 messages 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 0 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present} 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 0 to shard default 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 0 to shard default 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@6b2b4d29, splicing queue 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@42e3da9d queue spliced 1 messages 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-1-frontend-datastore-testSnapshotOnRootOverwrite: refreshing backend for shard 1 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-cars-testSnapshotOnRootOverwrite#-1817756744], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:12,9 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] 09:32:12,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=default, waitUntilReady=true] 09:32:12,10 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-default-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:32:12,10 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:12,10 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testSnapshotOnRootOverwrite, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:32:12,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSnapshotOnRootOverwrite 09:32:12,118 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Sending AppendEntries to follower member-2-shard-cars-testSnapshotOnRootOverwrite: AppendEntries [leaderId=member-1-shard-cars-testSnapshotOnRootOverwrite, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-0-1, size=57}]]] 09:32:12,238 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-default-testSnapshotOnRootOverwrite: Failed to transfer leadership in 229.4 ms 09:32:12,238 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: In unpauseLeader 09:32:12,238 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSnapshotOnRootOverwrite: Failed to transfer leadership in 229.5 ms 09:32:12,238 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: In unpauseLeader 09:32:12,238 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: leader transfer failed - sending PoisonPill 09:32:12,238 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite (Leader): Leadership transfer expired 09:32:12,238 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: leader transfer failed - sending PoisonPill 09:32:12,238 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader): Leadership transfer expired 09:32:12,239 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSnapshotOnRootOverwrite 09:32:12,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSnapshotOnRootOverwrite (Leader)_slicer-id-820: Closing 09:32:12,239 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:32:12,239 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-default-testSnapshotOnRootOverwrite 09:32:12,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-default-testSnapshotOnRootOverwrite (Leader)_slicer-id-819: Closing 09:32:12,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-default-testSnapshotOnRootOverwrite: Aborting 0 pending queued transactions 09:32:12,239 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSnapshotOnRootOverwrite: All shards shutdown - sending PoisonPill to self 09:32:12,239 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testSnapshotOnRootOverwrite 09:32:12,240 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:32:12,240 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSnapshotOnRootOverwrite: shutting down 09:32:12,240 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSnapshotOnRootOverwrite: Closing 09:32:12,241 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite#-1665699741] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:12,241 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:12,241 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:12,246 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:12,246 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:12,246 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:12,246 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:12,253 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:12,259 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:12,259 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:13,254 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [1891161438011720342] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:13,274 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [1339944379520911183] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:13,279 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:13,279 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:13,280 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:13.282] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:13,290 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:13,290 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:13,292 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:13.293] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:13,314 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:13,319 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4172109602283673114] 09:32:13,319 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:13,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:13,320 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:13,321 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:13,321 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:13,333 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:13,333 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:13,333 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:13,338 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:13,346 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [5828164290785868159] 09:32:13,346 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:13,348 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:13,348 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:13,348 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:13,348 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:13,363 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:13,371 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4694392357120394565] 09:32:13,371 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:13,381 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:13,381 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:13,381 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:13,381 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:13,382 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:13,383 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:13,384 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:32:13,389 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:13,391 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-90795766] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,394 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:13,396 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:13,404 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:32:13,404 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:13,404 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2c93a1c2 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e4cb523 09:32:13,404 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4a733832 09:32:13,404 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:32:13,404 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,405 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:13,405 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037533404, None)) 09:32:13,405 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,405 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037533403,0,0)) 09:32:13,406 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e4cb523 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@132bed6 09:32:13,406 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@283e1535 09:32:13,407 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 09:32:13,408 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:13,408 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:13,408 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,409 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:13,409 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,409 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,409 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-1-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491] 09:32:13,409 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:32:13,410 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:32:13,410 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,410 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:32:13,410 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-166364226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,410 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#148678803] 09:32:13,413 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-notifier#-166364226 created and ready for shard:member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,414 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/shard-member-1:snapshot-read#148678803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:32:13,414 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:32:13,414 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithSingleShard is initialized 09:32:13,414 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 09:32:13,417 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#786065145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,418 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:13,419 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:13,420 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:13,421 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from null to Follower 09:32:13,422 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: no initial generation override, starting from 0 09:32:13,422 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:13,422 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@17a9ee72 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5c36d678 09:32:13,422 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6e214e16 09:32:13,423 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:32:13,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:13,423 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:13,423 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,423 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037533423, None)) 09:32:13,423 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037533422,0,0)) 09:32:13,423 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5c36d678 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@d2d71e3 09:32:13,423 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} started 09:32:13,423 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@d8576e9 09:32:13,424 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:13,424 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:13,424 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,424 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:13,424 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: updated peer member-1-shard-cars-testWriteTransactionWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard on shard member-2-shard-cars-testWriteTransactionWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-2023657432] 09:32:13,424 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:32:13,424 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#-2023657432] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,424 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1717726982 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:13,424 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#395155640] 09:32:13,425 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1717726982] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,425 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#395155640] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:13,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:32:13,425 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:32:13,425 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,425 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard 09:32:13,425 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from null to Follower 09:32:13,474 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-90795766] 09:32:13,474 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:32:13,474 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491] 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:13,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:32:13,732 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:32:13,734 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Follower to Candidate 09:32:14,3 AM [cluster-test-shard-dispatcher-28] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:14,31 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:14,31 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,32 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 2 09:32:14,271 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:14,271 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,271 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 3 09:32:14,332 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:14,332 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,333 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithSingleShard/member-3-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,340 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard to akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,401 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:14,541 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:14,541 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,541 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 4 09:32:14,541 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Cluster state: [] 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:14,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:14,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:14,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithSingleShard] 09:32:14,544 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:14,545 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:14,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:32:14,545 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [] 09:32:14,546 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:32:14,546 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:14,546 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:14,546 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:14,546 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-1-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 09:32:14,546 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:32:14,546 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard: Closing 09:32:14,546 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,547 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done false 09:32:14,547 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:14,547 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:14,547 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:14,547 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:32:14,547 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:32:14,547 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:14,580 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:14,581 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:14,581 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:14,581 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:14,581 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#786065145] 09:32:14,581 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 09:32:14,581 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:14,581 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:14,581 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:14,581 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@4123890b 09:32:14,581 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:14,581 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:14,581 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=12] 09:32:14,581 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard, primaryVersion=12] 09:32:14,581 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1ad5599d, attempting to connect 09:32:14,583 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 09:32:14,583 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=45971, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:14,584 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60fed777 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:14,584 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2302/0x000000080170df70@694c9707 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE}, enqueuedTicks=740874795746} 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE}} 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message size: 509 bytes, total slices to send: 1 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message does not need to be sliced - sending original message 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2182e120 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1e010f9a superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3d0560f5, splicing queue 09:32:14,585 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1e010f9a queue spliced 0 messages 09:32:14,586 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.588 ms 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 ready 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 to ready 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct canCommit 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 validated 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct preCommit 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 candidate ready 09:32:14,588 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: canCommit - messages: [] 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 prepared 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 initiating direct commit 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 246 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 09:32:14,589 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 submitted to persistence 09:32:14,590 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}]]] 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 1 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=246} 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 1 09:32:14,592 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:14,593 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=246}] 09:32:14,593 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithSingleShard status sync done true 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 0, nextIndex: 1 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 0 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5d8d674c}, purged=MutableUnsignedLongSet{size=0}} 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#911647390] with name member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 09:32:14,593 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#911647390] 09:32:14,593 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:14,593 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:14,594 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:32:14,594 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 81.10 μs 09:32:14,594 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#911647390] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 0 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 0 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 0 data Simple{size=246} 09:32:14,594 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,595 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@224d43e7}, purged=MutableUnsignedLongSet{size=0}} 09:32:14,595 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,595 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,595 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2015097893] with name member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:14,595 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=0 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE}, enqueuedTicks=740874795746, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=4, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2015097893] 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4f32ff20 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 56.17 μs 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} allocated sequence 1 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@14a57d67 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher#2015097893] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:14,595 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]}, enqueuedTicks=740888833884} 09:32:14,595 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:14,595 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:14,595 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:14,596 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@14b93b5 09:32:14,596 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=740889347933} 09:32:14,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 09:32:14,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 09:32:14,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:14,597 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 09:32:14,598 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:14,599 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: Message size: 391 bytes, total slices to send: 1 09:32:14,599 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: Message does not need to be sliced - sending original message 09:32:14,599 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 09:32:14,600 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:14,600 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=740889347933, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0}} 09:32:14,600 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0} 09:32:14,600 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@10447030: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:14,601 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:14,601 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:14,601 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:14,601 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-90795766] 09:32:14,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:14,601 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} shard 1 09:32:14,601 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-0 09:32:14,601 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:14,601 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#943204867], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@18737ce1 09:32:14,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42faa700] 09:32:14,601 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:14,601 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@42faa700] 09:32:14,601 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5698f436, attempting to connect 09:32:14,601 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} is not yet known 09:32:14,602 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=6850, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:14,602 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@457e5eda instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2463c795 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2463c795 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#943204867], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@10447030 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5cb1cbfb 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@12686558 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@280d47ef, splicing queue 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@12686558 queue spliced 0 messages 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 587.5 μs 09:32:14,602 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0} 09:32:14,602 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04: creating a new transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:32:14,603 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:32:14,603 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=OPEN} allocated sequence 0 09:32:14,603 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04 readied transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-0 09:32:14,603 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@601c2418 09:32:14,603 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=1, protocol=SIMPLE}, enqueuedTicks=740896395603} 09:32:14,603 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=1, protocol=SIMPLE}} 09:32:14,603 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message size: 417 bytes, total slices to send: 1 09:32:14,603 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message does not need to be sliced - sending original message 09:32:14,603 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,603 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: readying transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 ready 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: transitioned member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 to ready 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct canCommit 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Validating transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 validated 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct preCommit 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Preparing transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 candidate ready 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - candidate rootPath: / 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: canCommit - messages: [] 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 prepared 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 initiating direct commit 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Starting commit for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 213 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 submitted to persistence 09:32:14,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithSingleShard, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}]]] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Number of entries to be appended = 2 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Append entry to log Simple{size=213} 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Log size is now 3 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithSingleShard, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65}] 09:32:14,605 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=213}] 09:32:14,605 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithSingleShard updated: matchIndex: 2, nextIndex: 3 09:32:14,605 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:32:14,605 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 1 09:32:14,605 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:32:14,605 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: replication of member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@aa2df02 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: finished purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Setting last applied to 2 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: Resuming commit of transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1: commit - successfulFromPrevious: [] 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 09:32:14,606 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:32:14,606 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 36.42 μs 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithSingleShard: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:32:14,606 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Commit index set to 2 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 1 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 1 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, size=65} 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,606 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Setting last applied to 2 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 1 to 2 09:32:14,606 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]}, enqueuedTicks=740888833884, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1}} 09:32:14,606 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying state for log index 2 data Simple{size=213} 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE}: purge completed 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=DONE} 09:32:14,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Applying foreign transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@60fed777 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=1, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=1, protocol=SIMPLE}, enqueuedTicks=740896395603, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=0}} 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED} directCommit completed 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=FLUSHED}: initiating purge 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} allocated sequence 1 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4b652802 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185]}, enqueuedTicks=740900487342} 09:32:14,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:14,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithSingleShard: publishChanges: rootPath: / 09:32:14,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: fakeSnapshot purging log to 1 for term 4 09:32:14,607 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Persisting ApplyJournalEntries with index=2 09:32:14,607 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 09:32:14,607 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithSingleShard-DTCL-publisher: 67.04 μs 09:32:14,607 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0 09:32:14,607 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 0 09:32:14,607 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1324/0x000000080135cf70@759c1835 09:32:14,607 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=740900991151} 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithSingleShard: purging transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1 09:32:14,609 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1, size=66}] 09:32:14,610 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 09:32:14,610 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]}, enqueuedTicks=740900991151, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true}} 09:32:14,610 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}] completed with ExistsTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=0, exists=true} 09:32:14,610 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6d4beb04: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 09:32:14,610 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0 09:32:14,610 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 0 09:32:14,610 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@1653c6a8 09:32:14,610 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=740903954922} 09:32:14,611 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithSingleShard: allocating new open snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1 09:32:14,615 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 09:32:14,615 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: Message size: 370 bytes, total slices to send: 1 09:32:14,615 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard_slicer-id-822: Message does not need to be sliced - sending original message 09:32:14,616 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 09:32:14,616 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-959167185], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=740903954922, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0}} 09:32:14,616 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=0} 09:32:14,616 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@10447030: creating a new snapshot member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:14,616 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1 for transaction member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0 09:32:14,616 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard#-90795766] 09:32:14,617 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:32:14,617 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491] 09:32:14,617 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491] 09:32:14,668 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:14,668 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:14,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-825: Closing 09:32:14,668 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:14,668 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:14,668 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:14,668 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:14,673 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:14,673 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:14,673 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:14,673 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:14,675 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:14,683 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:14,683 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:14,683 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:32:14,683 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:14,683 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:14,683 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:14,683 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:15,681 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4172109602283673114] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:15,701 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [5828164290785868159] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:15,706 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:15,706 AM [cluster-test-akka.actor.default-dispatcher-25] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:15.707] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:15,722 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:15,727 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://reinstated-member2@127.0.0.1:2559] with UID [-2331880794727906241] 09:32:15,727 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:15,728 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:15,728 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - Started up successfully 09:32:15,729 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:15,729 AM [reinstated-member2-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://reinstated-member2@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:15,740 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,744 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,746 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithSingleShard 09:32:15,757 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} 09:32:15,757 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} allocated sequence 1 09:32:15,757 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@29da70e2 09:32:15,757 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}, enqueuedTicks=742050694186} 09:32:15,757 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}} 09:32:15,757 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message size: 286 bytes, total slices to send: 1 09:32:15,757 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message does not need to be sliced - sending original message 09:32:15,757 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-3-0} 09:32:15,757 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} allocated sequence 1 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@7812bf31 09:32:15,758 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}, enqueuedTicks=742051361258} 09:32:15,758 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}} 09:32:15,758 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message size: 286 bytes, total slices to send: 1 09:32:15,758 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message does not need to be sliced - sending original message 09:32:15,758 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-4-0} 09:32:15,758 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2463c795 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#943204867]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@630cfbd2 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:15,758 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@457e5eda aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#943204867], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@254b0e54 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1} 09:32:15,758 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2463c795 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#943204867]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@387fbe39 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithSingleShard/member-1-shard-cars-testWriteTransactionWithSingleShard#-271865491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:15,758 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40694828 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 09:32:15,758 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@6145ac75 09:32:15,758 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@636eaa0e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}, enqueuedTicks=742052247682} 09:32:15,758 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: slice: identifier: member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-959167185], modifications=0, protocol=ABORT}} 09:32:15,759 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message size: 286 bytes, total slices to send: 1 09:32:15,759 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithSingleShard_slicer-id-823: Message does not need to be sliced - sending original message 09:32:15,759 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-1-0} 09:32:15,759 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@691fba82 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} 09:32:15,759 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testSnapshotOnRootOverwrite-fe-0-txn-1-0, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1068355750]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@48b18a1c 09:32:15,759 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} 09:32:15,759 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testSnapshotOnRootOverwrite: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testSnapshotOnRootOverwrite, generation=0}, cookie=0, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSnapshotOnRootOverwrite/member-1-shard-default-testSnapshotOnRootOverwrite#1949617125], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=0, shard=default, dataTree=present}} scheduling timeout in 30 seconds 09:32:15,759 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b2b65f6 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} 09:32:15,759 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} allocated sequence 1 09:32:15,759 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@76dc7152 09:32:15,759 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3ac145c6 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=ABORT}, enqueuedTicks=742052897874} 09:32:15,759 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: slice: identifier: member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1661517728], modifications=0, protocol=ABORT}} 09:32:15,759 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message size: 291 bytes, total slices to send: 1 09:32:15,759 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithSingleShard_slicer-id-809: Message does not need to be sliced - sending original message 09:32:15,759 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithSingleShard-fe-0-txn-1-0} 09:32:15,759 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:15,764 AM [reinstated-member2-akka.actor.default-dispatcher-15] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:15,764 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithSingleShard 09:32:15,764 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,764 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Creating local shard: member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,764 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:32:15,765 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@60a4f3da to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2ea55305 09:32:15,765 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,766 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6ba7c8b5 09:32:15,767 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithSingleShard, 0, 1714037535765, None)) 09:32:15,767 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037535764,0,0)) 09:32:15,767 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@389411e9 09:32:15,767 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2ea55305 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5a06e3c6 09:32:15,768 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} started 09:32:15,768 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:15,768 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,768 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,768 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithSingleShard, persistent : true 09:32:15,769 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard#1068758186] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,769 AM [reinstated-member2-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#831012904] 09:32:15,769 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1348491530 created and ready for shard:member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,770 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard-notifier#1348491530] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,770 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-people-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#831012904] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,770 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithSingleShard with journal batch size 1 09:32:15,772 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 09:32:15,772 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:15,773 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [people] 09:32:15,773 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithSingleShard is initialized 09:32:15,773 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithSingleShard, persistent : true 09:32:15,773 AM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,774 AM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1448798599 created and ready for shard:member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,774 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithSingleShard with journal batch size 1 09:32:15,774 AM [reinstated-member2-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard-notifier#1448798599] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,774 AM [reinstated-member2-akka.actor.default-dispatcher-17] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1281014373] 09:32:15,774 AM [reinstated-member2-akka.actor.default-dispatcher-17] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard/shard-member-2:snapshot-read#-1281014373] to Actor[akka://reinstated-member2/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=null 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithSingleShard 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 0, size: 246 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 0 to 0 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@b41d733}, purged=MutableUnsignedLongSet{size=0}} 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from -1 to 0 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 1, size: 65 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received ReplicatedLogEntry for recovery: index: 2, size: 213 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://reinstated-member2/deadLetters] 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Received apply journal entries for recovery, applying to state: 1 to 2 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:32:15,775 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Purged transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-0-1 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: starting log recovery batch with max size 1 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, history=0, cookie=1} 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithSingleShard: Committed transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-0-txn-2-1 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Moving last applied index from 0 to 2 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://reinstated-member2/deadLetters] 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Recovery completed in 888.4 μs - Switching actor to Follower - last log index = 2, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 3 09:32:15,776 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Initializing shard [cars] 09:32:15,776 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithSingleShard is initialized 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Set currentTerm=5, votedFor=member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Starting new election term 5 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithSingleShard (Follower): Closing 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,776 AM [reinstated-member2-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithSingleShard is no longer the leader 09:32:15,776 AM [reinstated-member2-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from null to Follower 09:32:15,776 AM [reinstated-member2-akka.actor.default-dispatcher-17] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:32:15,776 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Received TimeoutNow - switching to Candidate 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Election: Candidate has following voting peers: [] 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Set currentTerm=1, votedFor=member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Starting new election term 1 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithSingleShard (Follower): Closing 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithSingleShard is no longer the leader 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: became leader with frontend state for [member-2-frontend-datastore-testWriteTransactionWithSingleShard] 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from null to Follower 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Follower to Candidate 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderId=member-2-shard-cars-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate): Received ElectionTimeout 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Leader): Election: Leader has following peers: [] 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 1 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:32:15,777 AM [reinstated-member2-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-cars-testWriteTransactionWithSingleShard from Candidate to Leader 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: became leader with frontend state for [] 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , received role change from Candidate to Leader 09:32:15,777 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithSingleShard , registered listener akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderId=member-2-shard-people-testWriteTransactionWithSingleShard, leaderPayloadVersion=12] 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Received role changed for member-2-shard-people-testWriteTransactionWithSingleShard from Candidate to Leader 09:32:15,777 AM [reinstated-member2-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All Shards are ready - data store testWriteTransactionWithSingleShard is ready 09:32:15,818 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] 09:32:15,818 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: findLocalShard : cars 09:32:15,818 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869] 09:32:15,819 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7901d137: creating a new snapshot member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 09:32:15,819 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 09:32:15,819 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:15,819 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:15,819 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] 09:32:15,819 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:15,819 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=0} shard 1 09:32:15,819 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6ab53c50] 09:32:15,819 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-0 09:32:15,819 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:15,819 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#808283918], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@291b2166 09:32:15,819 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} scheduling timeout in 30 seconds 09:32:15,819 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://reinstated-member2@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@6ab53c50] 09:32:15,820 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6c7ef8db, attempting to connect 09:32:15,820 AM [reinstated-member2-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: retiring state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0}, nanosAgo=43242585, purgedHistories=MutableUnsignedLongSet{size=0}}, outdated by request from client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:32:15,820 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, nanosAgo=9930, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:32:15,820 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1f2469e3 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4ca3a860 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4ca3a860 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#808283918], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7901d137 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@683f0b66 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6a896498 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3b383101, splicing queue 09:32:15,821 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6a896498 queue spliced 0 messages 09:32:15,822 AM [reinstated-member2-akka.actor.default-dispatcher-12] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 740.1 μs 09:32:15,822 AM [reinstated-member2-akka.actor.default-dispatcher-12] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0} 09:32:15,822 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1} 09:32:15,822 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,822 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,822 AM [reinstated-member2-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithSingleShard - waiting on 2 shards 09:32:15,822 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: onShutDown 09:32:15,822 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: onShutDown 09:32:15,822 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@31dfd07b 09:32:15,822 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActor$4@4ef654f2 09:32:15,822 AM [reinstated-member2-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@4ef654f2 09:32:15,822 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActor$4@31dfd07b 09:32:15,823 AM [reinstated-member2-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithSingleShard 09:32:15,823 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] MessageSlicer - member-2-shard-people-testWriteTransactionWithSingleShard (Leader)_slicer-id-830: Closing 09:32:15,823 AM [reinstated-member2-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithSingleShard 09:32:15,823 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testWriteTransactionWithSingleShard (Leader)_slicer-id-829: Closing 09:32:15,823 AM [reinstated-member2-shard-dispatcher-11] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:15,823 AM [reinstated-member2-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithSingleShard: Aborting 0 pending queued transactions 09:32:15,823 AM [reinstated-member2-akka.actor.default-dispatcher-5] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithSingleShard: All shards shutdown - sending PoisonPill to self 09:32:15,823 AM [reinstated-member2-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithSingleShard 09:32:15,824 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:15,824 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:15,824 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:204) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:32:15,824 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:15,824 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithSingleShard, generation=0} 09:32:15,824 AM [main] [WARN] ActorUtils - ShardManager for testWriteTransactionWithSingleShard data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:32:15,824 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:15,825 AM [reinstated-member2-akka.actor.default-dispatcher-17] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithSingleShard: shutting down 09:32:15,825 AM [reinstated-member2-akka.actor.default-dispatcher-17] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithSingleShard: Closing 09:32:15,825 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,825 AM [reinstated-member2-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard#554360473] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,825 AM [reinstated-member2-akka.actor.default-dispatcher-17] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://reinstated-member2/user/$a#808283918] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://reinstated-member2/user/$a#808283918] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,825 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:15,825 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:15,827 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:15,832 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:15,832 AM [reinstated-member2-akka.actor.default-dispatcher-18] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:15,833 AM [reinstated-member2-akka.actor.default-dispatcher-17] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:15.834] [reinstated-member2-akka.actor.internal-dispatcher-3] [Cluster(akka://reinstated-member2)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:15,845 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:15,851 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1047026111367026547] 09:32:15,852 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:15,854 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:15,854 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:15,854 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:15,854 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:15,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:15,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:15,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:15,869 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:15,878 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3855491897281117957] 09:32:15,878 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:15,880 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:15,881 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:15,899 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:15,912 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:15,913 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:15,926 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1419000341918707540] 09:32:15,927 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:15,931 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:15,931 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:15,932 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:15,932 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:15,944 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,946 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,949 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,956 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,956 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#1447121560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,960 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,960 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 09:32:15,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:32:15,960 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:15,960 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@59aa8081 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ce90c43 09:32:15,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@48b790a1 09:32:15,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1714037535960, None)) 09:32:15,961 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037535959,0,0)) 09:32:15,961 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@24710bca 09:32:15,961 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@ce90c43 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2bd2bba4 09:32:15,961 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 09:32:15,961 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,963 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:15,965 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:15,965 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,973 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,973 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,978 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:15,979 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:15,983 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: no initial generation override, starting from 0 09:32:15,983 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,983 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:32:15,983 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Creating local shard: member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,983 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5c88a879 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6ddc5df5 09:32:15,983 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a82f4d9 09:32:15,984 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:15,984 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,984 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, 0, 1714037535984, None)) 09:32:15,984 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037535983,0,0)) 09:32:15,984 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6ddc5df5 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@19980314 09:32:15,984 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@f1b9f1a 09:32:15,985 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} started 09:32:15,985 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:15,985 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,985 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,985 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,985 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-884666438] 09:32:15,985 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,985 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: updated peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader on shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223] 09:32:15,986 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 09:32:15,986 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, persistent : true 09:32:15,986 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 09:32:15,986 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with journal batch size 1 09:32:15,986 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,986 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1190280137] 09:32:15,986 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#26173504 created and ready for shard:member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,987 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#26173504] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,987 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-1:snapshot-read#1190280137] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:15,987 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:15,987 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 09:32:15,987 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 09:32:15,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:32:15,987 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 09:32:15,987 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,988 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 09:32:15,990 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-884666438] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,990 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#103879173] 09:32:15,991 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#2088220000] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:15,991 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader-notifier#2088220000 created and ready for shard:member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:15,992 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:15,992 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Initializing shard [cars] 09:32:15,992 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is initialized 09:32:15,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:15,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:32:15,992 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,992 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from null to Follower 09:32:15,992 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:15,992 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from null to Follower 09:32:15,992 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader/shard-member-2:snapshot-read#103879173] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:16,35 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#1447121560] 09:32:16,35 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: findLocalShard : cars 09:32:16,35 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223] 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 1 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:32:16,273 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 09:32:16,273 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 09:32:16,532 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:16,532 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,532 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 2 09:32:16,586 AM [cluster-test-shard-dispatcher-19] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:16,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:16,822 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,822 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 3 09:32:16,872 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:16,873 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,873 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,873 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-3-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,881 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader to akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:16,945 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:17,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:17,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,102 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 4 09:32:17,102 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Cluster state: [] 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=null 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:17,105 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:17,105 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:17,105 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:32:17,105 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: became leader with frontend state for [] 09:32:17,106 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Candidate to Leader 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=12] 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Candidate to Leader 09:32:17,106 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader status sync done false 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderPayloadVersion=12] 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:17,107 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All Shards are ready - data store testTransactionWithCreateTxFailureDueToNoLeader is ready 09:32:17,108 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader updated: matchIndex: -1, nextIndex: 0 09:32:17,108 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:17,141 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2819f81c: creating a new snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 09:32:17,142 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 09:32:17,142 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:17,142 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:17,142 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] 09:32:17,142 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:17,142 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=0} shard 1 09:32:17,142 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0 09:32:17,142 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:17,142 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=12] 09:32:17,142 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@33ad18ec 09:32:17,142 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} scheduling timeout in 2 seconds 09:32:17,142 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, primaryVersion=12] 09:32:17,143 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1f4a9944, attempting to connect 09:32:17,144 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} is not yet known 09:32:17,144 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, nanosAgo=11920, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:32:17,146 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@56d49622 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@22c47e20 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@22c47e20 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@2f98fb21 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1995539714 nanoseconds 09:32:17,146 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@566ba223 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=743435680093} 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2819f81c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@57dfe3b0 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7e91b53 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@6f40a97c, splicing queue 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7e91b53 queue spliced 0 messages 09:32:17,147 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.050 ms 09:32:17,148 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: allocating new open snapshot member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1 09:32:17,148 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-833: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 09:32:17,148 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-833: Message size: 239 bytes, total slices to send: 1 09:32:17,148 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-833: Message does not need to be sliced - sending original message 09:32:17,155 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 09:32:17,155 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=743435680093, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0}} 09:32:17,155 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=0} 09:32:17,156 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:17,156 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Leader)_slicer-id-835: Closing 09:32:17,156 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 09:32:17,156 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,157 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:17,157 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:17,165 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:17,165 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:17,165 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:17,165 AM [cluster-test-akka.actor.default-dispatcher-22] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:17,167 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:17,185 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:32:17,285 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,285 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2819f81c: creating a new transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 09:32:17,285 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 09:32:17,285 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:17,286 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@2819f81c readied transaction member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-0 09:32:17,286 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6f35ed9 09:32:17,286 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:17,286 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@566ba223 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=1, protocol=SIMPLE}, enqueuedTicks=743579516130} 09:32:17,286 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-832: slice: identifier: member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=1, protocol=SIMPLE}} 09:32:17,286 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-832: Message size: 401 bytes, total slices to send: 1 09:32:17,286 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader_slicer-id-832: Message does not need to be sliced - sending original message 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : true 09:32:17,287 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Received ElectionTimeout - switching to Candidate 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader] 09:32:17,447 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 5 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Follower): Closing 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader is no longer the leader 09:32:17,448 AM [cluster-test-shard-dispatcher-19] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:17,448 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader , received role change from Follower to Candidate 09:32:17,449 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderId=null, leaderPayloadVersion=-1] 09:32:17,449 AM [cluster-test-shard-dispatcher-19] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:17,449 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received role changed for member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader from Follower to Candidate 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@566ba223 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@6431740d, splicing queue 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1836684822 nanoseconds 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@566ba223 queue spliced 1 messages 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: refreshing backend for shard 1 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:17,449 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:17,450 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] 09:32:17,450 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:17,450 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 09:32:17,596 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:17,596 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=6, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,596 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 6 09:32:17,597 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,666 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 09:32:17,666 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 09:32:17,666 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:17,667 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:17,667 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:17,667 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:17,716 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:17,716 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=7, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,716 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 7 09:32:17,717 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,856 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:17,856 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=8, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,856 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 8 09:32:17,856 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,987 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:17,987 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=9, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:17,987 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 9 09:32:17,987 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,166 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,167 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=10, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,167 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 10 09:32:18,167 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,175 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1047026111367026547] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:18,177 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1047026111367026547] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:18,326 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,326 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=11, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,326 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 11 09:32:18,326 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,476 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,476 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=12, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,476 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 12 09:32:18,477 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,647 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,647 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=13, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,647 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 13 09:32:18,647 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,686 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:18,686 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:18,686 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:18,686 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] 09:32:18,686 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:18,686 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 09:32:18,776 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,776 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=14, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,776 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 14 09:32:18,776 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,906 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Received ShardNotInitializedTimeout message for shard cars 09:32:18,907 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Returning NoShardLeaderException for shard cars 09:32:18,907 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:18,907 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:18,907 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:18,908 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:18,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:18,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=15, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:18,946 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 15 09:32:18,946 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,117 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:19,117 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=16, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,117 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 16 09:32:19,117 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,156 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 09:32:19,156 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} is empty 09:32:19,156 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} 09:32:19,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:19,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 09:32:19,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:19,256 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Received ElectionTimeout 09:32:19,257 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Set currentTerm=17, votedFor=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,257 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Starting new election term 17 09:32:19,257 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=1, protocol=SIMPLE}, enqueuedTicks=743579516130, sessionId=0, txSequence=1} 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: not scheduling timeout on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED} directCommit completed 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=FLUSHED}: initiating purge 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#888021220]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@143b2aed 09:32:19,306 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 09:32:19,307 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999954290 nanoseconds 09:32:19,307 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:32:19,307 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,307 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader - waiting on 1 shards 09:32:19,307 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: onShutDown 09:32:19,308 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,308 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader: Aborting 0 pending queued transactions 09:32:19,308 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithCreateTxFailureDueToNoLeader: All shards shutdown - sending PoisonPill to self 09:32:19,308 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithCreateTxFailureDueToNoLeader 09:32:19,308 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:19,308 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0} 09:32:19,308 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: shutting down 09:32:19,308 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader#540389691] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:19,308 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithCreateTxFailureDueToNoLeader data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:32:19,309 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:19,308 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: Closing 09:32:19,309 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#888021220] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#888021220] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:19,309 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:19,309 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:20,324 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3855491897281117957] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [04/25/2024 09:32:20.333] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config [WARN] [04/25/2024 09:32:20.329] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1073)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:20.329] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1073)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:20,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:20,349 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:32:20.352] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:20,371 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:20,386 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-5965137404892615354] 09:32:20,386 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:20,387 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:20,387 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:20,395 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:20,395 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:20,400 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:20,400 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:20,400 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:20,401 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:20,405 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2579948104866753486] 09:32:20,405 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:20,407 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:20,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:20,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:20,407 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:20,422 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:20,432 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4226287671578302886] 09:32:20,432 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:20,433 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:20,433 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:20,435 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:20,435 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:20,443 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:20,445 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:20,446 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,457 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-771704135] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,457 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:20,457 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:20,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 09:32:20,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:20,458 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@33a2839a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@62ff5670 09:32:20,459 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1cb79f7a 09:32:20,459 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1714037540459, None)) 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,459 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037540458,0,0)) 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,459 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@62ff5670 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2fa87e7e 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,459 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@751413cd 09:32:20,459 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 09:32:20,459 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:20,461 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:20,461 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561] 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,462 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018] 09:32:20,462 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 09:32:20,462 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Shard created : member-1-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 09:32:20,463 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,463 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,463 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:32:20,463 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Starting recovery for member-1-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:32:20,463 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1440706160] 09:32:20,463 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#-1440706160] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,464 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,465 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1741391544 created and ready for shard:member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,465 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-1741391544] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,465 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:20,465 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:20,465 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:20,465 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 09:32:20,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1163953555 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 09:32:20,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#1163953555] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:32:20,466 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,466 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 09:32:20,468 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:20,468 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1487982196] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,468 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:20,469 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,469 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,469 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:32:20,469 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,469 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 09:32:20,469 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#561895835] 09:32:20,469 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-1:snapshot-read#561895835] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,471 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:20,471 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: no initial generation override, starting from 0 09:32:20,471 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:20,471 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,471 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2b112c49 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@71af209f 09:32:20,471 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@254f10d6 09:32:20,471 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, 0, 1714037540471, None)) 09:32:20,471 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Creating local shard: member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037540470,0,0)) 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5303eab2 09:32:20,472 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@71af209f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35c235a1 09:32:20,472 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} started 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1901123552] 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,472 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: updated peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards on shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-609278198] 09:32:20,473 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-people-testChainedTransactionFailureWithMultipleShards, persistent : true 09:32:20,473 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, persistent : true 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards#-609278198] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1901123552] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-856436207 created and ready for shard:member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-830488393 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards-notifier#-856436207] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#13280601] 09:32:20,473 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-people-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:32:20,473 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1994483943] 09:32:20,473 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with journal batch size 1 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:20,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:20,474 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,474 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,474 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,474 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from null to Follower 09:32:20,474 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [people] 09:32:20,474 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards is initialized 09:32:20,474 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,477 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:20,478 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards-notifier#-830488393] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,478 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#13280601] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,478 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards/shard-member-2:snapshot-read#1994483943] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:20,482 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Initializing shard [cars] 09:32:20,482 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards is initialized 09:32:20,482 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-people-testChainedTransactionFailureWithMultipleShards from null to Follower 09:32:20,483 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards 09:32:20,483 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards from null to Follower 09:32:20,522 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-771704135] 09:32:20,522 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : cars 09:32:20,522 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561] 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:20,738 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:32:20,739 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,739 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,739 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 09:32:20,739 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 09:32:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:32:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 09:32:20,768 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 1 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards is no longer the leader 09:32:20,769 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Follower to Candidate 09:32:20,769 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Follower to Candidate 09:32:21,10 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:21,28 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,28 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,28 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 09:32:21,30 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:21,38 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,38 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,38 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 2 09:32:21,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,348 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,348 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 09:32:21,358 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,358 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,358 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 3 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,409 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,410 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,410 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,410 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:21,410 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,410 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-3-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,418 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,449 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:21,598 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,598 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,598 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 09:32:21,598 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,600 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithMultipleShards] 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 09:32:21,602 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:21,602 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:21,603 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done false 09:32:21,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:32:21,604 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:21,628 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Received ElectionTimeout 09:32:21,629 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,629 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Starting new election term 4 09:32:21,629 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Cluster state: [] 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=null 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,631 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:21,632 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testChainedTransactionFailureWithMultipleShards] 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: became leader with frontend state for [] 09:32:21,633 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testChainedTransactionFailureWithMultipleShards , received role change from Candidate to Leader 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received role changed for member-1-shard-people-testChainedTransactionFailureWithMultipleShards from Candidate to Leader 09:32:21,633 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 09:32:21,634 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,634 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testChainedTransactionFailureWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,634 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderPayloadVersion=12] 09:32:21,634 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:21,634 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:21,634 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All Shards are ready - data store testChainedTransactionFailureWithMultipleShards is ready 09:32:21,634 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-people-testChainedTransactionFailureWithMultipleShards status sync done false 09:32:21,634 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testChainedTransactionFailureWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:32:21,635 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:21,679 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-771704135] 09:32:21,679 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: findLocalShard : people 09:32:21,679 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018] 09:32:21,680 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c 09:32:21,681 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 569343289, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@8c49f0a} 09:32:21,681 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c changed state from IDLE to TX_OPEN 09:32:21,681 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 09:32:21,681 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard people 09:32:21,681 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard people 09:32:21,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1487982196] 09:32:21,682 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 1 09:32:21,682 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:21,682 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:32:21,682 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:21,682 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 09:32:21,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:21,682 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard cars 09:32:21,682 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard cars 09:32:21,682 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1487982196] 09:32:21,682 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} shard 2 09:32:21,682 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:32:21,682 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:32:21,682 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 09:32:21,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@4852a4a4 09:32:21,683 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@55b45aec 09:32:21,683 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c changed state from TX_OPEN to IDLE 09:32:21,683 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:32:21,683 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c readied transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-0 09:32:21,683 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@26c29fc5 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} allocated sequence 1 09:32:21,683 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, primaryVersion=12] 09:32:21,683 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@51ed6ad5 09:32:21,683 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7b9c515f, attempting to connect 09:32:21,683 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@290395ac, attempting to connect 09:32:21,686 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 09:32:21,686 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=13410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:21,686 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} is not yet known 09:32:21,686 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, nanosAgo=4770, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:21,688 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:21,688 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:21,688 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=people, dataTree=absent} 09:32:21,688 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@bec3be9 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6dbdaeb5 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7517ef5a transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=747975398779} 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@6dbdaeb5 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@d5a9ee2 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7517ef5a transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}, enqueuedTicks=747976460666} 09:32:21,689 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message size: 406 bytes, total slices to send: 1 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message does not need to be sliced - sending original message 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@6697f7ad 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7517ef5a transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=747976747171} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message size: 298 bytes, total slices to send: 1 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message does not need to be sliced - sending original message 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3eb2333e 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@20ed5ad3 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7ada9f83, splicing queue 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@20ed5ad3 queue spliced 0 messages 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-people-testChainedTransactionFailureWithMultipleShards#809330018], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=people, dataTree=absent}} in 1.635 ms 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=cars, dataTree=absent} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@681372c5 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@50f34732 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:21,690 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1a02a43d transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=747976247313} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@50f34732 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 for transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=OPEN} allocated sequence 0 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@763ece61 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1a02a43d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}, enqueuedTicks=747976537858} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message size: 491 bytes, total slices to send: 1 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message does not need to be sliced - sending original message 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 1 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@12405017 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1a02a43d transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=747976819722} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: slice: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, message: Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message size: 299 bytes, total slices to send: 1 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards_slicer-id-839: Message does not need to be sliced - sending original message 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@3b79510c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7814365d 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@41031894 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@69ae8d0d, splicing queue 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@41031894 queue spliced 0 messages 09:32:21,691 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=cars, dataTree=absent}} in 1.084 ms 09:32:21,692 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 09:32:21,692 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] LeaderFrontendState - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@5b95ce46 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: readying transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 ready 09:32:21,693 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 09:32:21,693 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people, operation=WRITE}}}]} 09:32:21,693 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: transitioned member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 to ready 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 is already in state READY (READY) 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 initiating canCommit 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@15067c7d 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Validating transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 validated 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 completed canCommit 09:32:21,694 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,694 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}] 09:32:21,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 09:32:21,695 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,696 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 09:32:21,697 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:21,697 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}, enqueuedTicks=747976460666, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:21,697 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=0} 09:32:21,698 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:21,698 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=747976747171, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:21,698 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} canCommit completed 09:32:21,709 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process}} 09:32:21,709 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY}, enqueuedTicks=747976537858, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process}} 09:32:21,709 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process} 09:32:21,709 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 09:32:21,710 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:32:21,710 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1894073585], modifications=0, protocol=THREE_PHASE}, enqueuedTicks=747976819722, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:32:21,710 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,710 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@52983d9e failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.resolveResult(VotingFuture.java:91) at org.opendaylight.controller.cluster.databroker.actors.dds.VotingFuture.voteNo(VotingFuture.java:67) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$canCommit$5(AbstractProxyTransaction.java:549) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$17(RemoteProxyTransaction.java:516) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013308d8@3685d3d5 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 2 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013308d8@3685d3d5 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7517ef5a transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748005051883} 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013308d8@4f907d19 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} allocated sequence 2 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED} sending request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1224/0x00000008013308d8@4f907d19 09:32:21,711 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1a02a43d transmitting entry ConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748005215775} 09:32:21,712 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} canCommit completed 09:32:21,716 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborting 09:32:21,716 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: aborting head of queue member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 in state member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,716 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1: abort - successfulFromPrevious: [] 09:32:21,716 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 aborted 09:32:21,716 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: request Envelope{sessionId=1, txSequence=3, message=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1894073585]}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,719 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,719 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}]]] 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Number of entries to be appended = 1 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Log size is now 1 09:32:21,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testChainedTransactionFailureWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748005051883, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionAbortSuccess{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} abort completed 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7ba04d8b 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=1} 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 3 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@63233251 09:32:21,720 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7517ef5a transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748014128618} 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Setting last applied to 0 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Moving last applied index from -1 to 0 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77} 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2178/0x000000080169b4b8@4f53b94c 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 09:32:21,721 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1 09:32:21,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:32:21,721 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionAbortRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748005215775, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=2, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:32:21,721 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 09:32:21,721 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 09:32:21,721 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendMetadata - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5406bd30}, purged=MutableUnsignedLongSet{size=0}} 09:32:21,721 AM [cluster-test-akka.actor.default-dispatcher-10] [ERROR] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during Abort. org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} abort completed 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR}: initiating purge 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@219fe298 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, cookie=2} 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=SEALED}: initiating purge 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} allocated sequence 3 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@30d7db77 09:32:21,722 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@1a02a43d transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=748015355818} 09:32:21,722 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, size=77}] 09:32:21,722 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithMultipleShards status sync done true 09:32:21,721 AM [cluster-test-shard-dispatcher-18] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 09:32:21,722 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}] 09:32:21,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:21,722 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:32:21,722 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@52983d9e is no longer running 09:32:21,722 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:21,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: purging transaction member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2 09:32:21,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 09:32:21,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 09:32:21,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:21,723 AM [cluster-test-shard-dispatcher-18] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}] 09:32:21,729 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,729 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,729 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 09:32:21,729 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 09:32:21,729 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 09:32:21,729 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585]} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 09:32:21,729 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1894073585]}, enqueuedTicks=747976247313, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0}} 09:32:21,729 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2}, sequence=0} 09:32:21,729 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:32:21,729 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:32:21,730 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,730 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-1-shard-cars-testChainedTransactionFailureWithMultipleShards#-1731471561] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1901123552] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards/member-2-shard-cars-testChainedTransactionFailureWithMultipleShards#1901123552] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:21,730 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testChainedTransactionFailureWithMultipleShards (Follower): Closing 09:32:21,730 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:32:21,731 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:32:21,731 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:21,731 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,731 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} 09:32:21,731 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Force-closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=0} java.lang.IllegalStateException: Client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0} has been shut down at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior.shutdown(AbstractDataStoreClientBehavior.java:113) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:148) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:21,731 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 09:32:21,731 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,731 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,731 AM [cluster-test-shard-dispatcher-18] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,731 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1894073585] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1894073585] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:21,731 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards - waiting on 2 shards 09:32:21,731 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: onShutDown 09:32:21,731 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: onShutDown 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Initiating leader transfer 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8285191 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=2} 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@8285191 09:32:21,732 AM [cluster-test-shard-dispatcher-18] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32132526 09:32:21,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1} 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@32132526 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Attempting to transfer leadership 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,732 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testChainedTransactionFailureWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithMultipleShards, generation=0}, history=1, cookie=1}, size=77}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-1, size=78}]]] 09:32:21,732 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1487982196] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-1487982196] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:21,749 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,839 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,839 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithMultipleShards-fe-0-chn-1-txn-0-2, size=78}]]] 09:32:21,958 AM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Failed to transfer leadership in 226.6 ms 09:32:21,958 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: In unpauseLeader 09:32:21,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: leader transfer failed - sending PoisonPill 09:32:21,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader): Leadership transfer expired 09:32:21,959 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithMultipleShards 09:32:21,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-842: Closing 09:32:21,959 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Failed to transfer leadership in 256.7 ms 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: In unpauseLeader 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: leader transfer failed - sending PoisonPill 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader): Leadership transfer expired 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testChainedTransactionFailureWithMultipleShards 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testChainedTransactionFailureWithMultipleShards (Leader)_slicer-id-843: Closing 09:32:21,989 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testChainedTransactionFailureWithMultipleShards: Aborting 0 pending queued transactions 09:32:21,990 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:32:21,990 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithMultipleShards 09:32:21,990 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: shutting down 09:32:21,991 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithMultipleShards: Closing 09:32:21,991 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-771704135] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithMultipleShards#-771704135] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:21,991 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:21,991 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:21,997 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:21,998 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:21,998 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:21,998 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:22,11 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:22,11 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:23,9 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-5965137404892615354] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:23,29 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2579948104866753486] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:23,33 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:23,33 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:23,36 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:23.037] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:23,42 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:23,42 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:23,45 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:23.045] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:23,68 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:23,73 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7789291332624578442] 09:32:23,73 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:23,74 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:23,74 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:23,75 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:23,75 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:23,84 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:23,84 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:23,84 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:23,86 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:23,91 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [770728117965576881] 09:32:23,91 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:23,93 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:23,93 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:23,103 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:23,103 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:23,112 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:23,133 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8992225380210711454] 09:32:23,134 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:23,135 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:23,135 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:23,142 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:23,142 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:23,147 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:23,148 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:23,150 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:32:23,151 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1155089234] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,152 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:23,161 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:23,164 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5c34bb6b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5313926e 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@3d37c4fd 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037543172, None)) 09:32:23,172 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037543171,0,0)) 09:32:23,172 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6c374a45 09:32:23,173 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5313926e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6a6f3c54 09:32:23,173 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:32:23,173 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:23,175 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:23,178 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:32:23,178 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:23,178 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,180 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,180 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,180 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-1-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] 09:32:23,180 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,181 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-1-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:32:23,181 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:23,181 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,181 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-374941747] 09:32:23,181 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#1712073238] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,181 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#-374941747] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,181 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,181 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,181 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-1-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] 09:32:23,182 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:32:23,182 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:32:23,182 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:32:23,183 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,183 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1477269237 created and ready for shard:member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,183 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#666843370] 09:32:23,183 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:23,183 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#1712073238 created and ready for shard:member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,186 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-notifier#-1477269237] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,213 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1f2469e3 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} 09:32:23,213 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:23,213 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=1, replyTo=Actor[akka://reinstated-member2/user/$a#808283918], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@445fdebc 09:32:23,213 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1} 09:32:23,213 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@4ca3a860 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} 09:32:23,213 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testWriteTransactionWithSingleShard-fe-1-txn-0-1, sequence=0, replyTo=Actor[akka://reinstated-member2/user/$a#808283918]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@285c865c 09:32:23,213 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithSingleShard, generation=1}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://reinstated-member2/user/shardmanager-testWriteTransactionWithSingleShard/member-2-shard-cars-testWriteTransactionWithSingleShard#472123869], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:23,214 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@56d49622 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@4c69ca3a 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1} has a successor, not scheduling timer 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#888021220], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@7f296f5d 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,214 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithCreateTxFailureDueToNoLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithCreateTxFailureDueToNoLeader/member-1-shard-cars-testTransactionWithCreateTxFailureDueToNoLeader#-862495223], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,214 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithCreateTxFailureDueToNoLeader-fe-0-txn-0-0} 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:32:23,214 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-people-testLeadershipTransferOnShutdown is initialized 09:32:23,214 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:23,214 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:23,214 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/shard-member-1:snapshot-read#666843370] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,214 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:23,214 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,214 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,214 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:23,214 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:23,215 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:32:23,215 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLeadershipTransferOnShutdown is initialized 09:32:23,215 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:32:23,215 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:23,216 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:32:23,216 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:23,216 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1464818602] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,218 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:32:23,218 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:23,218 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7d5613af to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@538780ac 09:32:23,218 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@57b81e04 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-2-shard-people-testLeadershipTransferOnShutdown 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,218 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,219 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037543218, None)) 09:32:23,219 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037543217,0,0)) 09:32:23,219 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@538780ac to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@56808742 09:32:23,219 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7aa1a91 09:32:23,219 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-2-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694] 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Shard created : member-2-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-2-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#44832787] 09:32:23,220 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:32:23,220 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1997213785 created and ready for shard:member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1903847824] 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#44832787] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-notifier#-1997213785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-1903847824] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,220 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#550381080 created and ready for shard:member-2-shard-people-testLeadershipTransferOnShutdown 09:32:23,221 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-notifier#550381080] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,221 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-194561729] 09:32:23,221 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/shard-member-2:snapshot-read#-194561729] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:23,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:23,221 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:23,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,221 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,221 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,221 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:32:23,221 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLeadershipTransferOnShutdown is initialized 09:32:23,221 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:23,221 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:23,221 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,222 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:23,222 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,223 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:32:23,223 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testLeadershipTransferOnShutdown is initialized 09:32:23,223 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:23,223 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:32:23,270 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1155089234] 09:32:23,270 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:32:23,270 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,475 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:32:23,475 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:23,504 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:32:23,505 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:23,505 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:23,505 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:32:23,505 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 09:32:23,734 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:23,734 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:23,735 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 09:32:23,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:23,784 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=2, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:32:23,784 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 2 09:32:24,55 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:24,55 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,55 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 09:32:24,74 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:24,74 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,74 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 3 09:32:24,95 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:24,95 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,96 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:24,106 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Peer address for peer member-3-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,106 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,154 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:32:24,255 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,256 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,256 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:32:24,256 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Follower to Candidate 09:32:24,257 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 09:32:24,257 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-people-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 09:32:24,257 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 09:32:24,257 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown#44832787] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,258 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 09:32:24,258 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-people-testLeadershipTransferOnShutdown from Candidate to Follower 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received ElectionTimeout - switching to Candidate 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=1, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 1 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,275 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,276 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:32:24,276 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:32:24,277 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,280 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1], my term is 3 09:32:24,280 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): In requestVote: RequestVote [term=1, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 3, votedFor: member-1-shard-cars-testLeadershipTransferOnShutdown, lastIndex: -1, lastTerm: -1 09:32:24,281 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): requestVote returning: RequestVoteReply [term=3, voteGranted=false] 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=3, voteGranted=false], my term is 1 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Term 3 in "RequestVoteReply [term=3, voteGranted=false]" message is greater than Candidate's term 1 - switching to Follower 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=3, votedFor=null 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Follower, election term: 3 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,281 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Follower 09:32:24,281 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Follower 09:32:24,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:24,324 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,325 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 09:32:24,325 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,325 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:24,326 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,326 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-people-testLeadershipTransferOnShutdown, member-3-shard-people-testLeadershipTransferOnShutdown] 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:32:24,327 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: became leader with frontend state for [] 09:32:24,327 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:32:24,328 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:24,328 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:24,328 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-people-testLeadershipTransferOnShutdown from Candidate to Leader 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown: Closing 09:32:24,328 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:24,328 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done false 09:32:24,330 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,330 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,364 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Received ElectionTimeout 09:32:24,364 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,364 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 4 09:32:24,364 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,364 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,365 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:24,365 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:32:24,365 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:24,365 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 3 - updating term 09:32:24,365 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:32:24,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:24,366 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,366 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:24,367 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:24,367 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:24,367 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-2-shard-cars-testLeadershipTransferOnShutdown, member-3-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,367 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,367 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 09:32:24,368 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 09:32:24,368 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:32:24,368 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,368 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,368 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,369 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,369 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:24,369 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:32:24,369 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:32:24,369 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,369 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,431 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1155089234] 09:32:24,431 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : people 09:32:24,431 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] 09:32:24,432 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:24,434 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:24,437 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLeadershipTransferOnShutdown 09:32:24,442 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1307629676] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,442 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:24,447 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,447 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-people-testLeadershipTransferOnShutdown status sync done true 09:32:24,448 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: no initial generation override, starting from 0 09:32:24,448 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:24,448 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@36f1fc9f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@34098645 09:32:24,448 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@54f1f559 09:32:24,448 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testLeadershipTransferOnShutdown, 0, 1714037544448, None)) 09:32:24,448 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Creating local shard: member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:24,449 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037544447,0,0)) 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4acb85d4 09:32:24,449 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@34098645 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@c4cc5ad 09:32:24,449 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} started 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1466745726] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1466745726] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-cars-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown on shard member-3-shard-cars-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1466745726] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:24,450 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-1-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#869839140] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1464818602] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-2-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#869839140] 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testLeadershipTransferOnShutdown with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown to akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: updated peer member-3-shard-people-testLeadershipTransferOnShutdown on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown on shard member-3-shard-people-testLeadershipTransferOnShutdown whose actor address is Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#869839140] 09:32:24,450 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 09:32:24,450 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:24,450 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-cars-testLeadershipTransferOnShutdown, persistent : true 09:32:24,450 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 09:32:24,450 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:24,450 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:24,450 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1464818602] 09:32:24,450 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 2 09:32:24,450 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:24,450 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:24,450 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:24,450 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198: creating a new transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:32:24,450 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:32:24,450 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:24,450 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:32:24,450 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198 readied transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-0 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-cars-testLeadershipTransferOnShutdown with journal batch size 1 09:32:24,450 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-3-shard-people-testLeadershipTransferOnShutdown, persistent : true 09:32:24,451 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testLeadershipTransferOnShutdown 09:32:24,451 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#-1155089234] 09:32:24,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,451 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: findLocalShard : cars 09:32:24,451 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:32:24,451 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 09:32:24,451 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=2, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@5a62fbf8 09:32:24,451 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:24,451 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@6fbe83cc 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71f257d9 09:32:24,451 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49be8d2b skipping transactions [1] 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@71f257d9 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:32:24,451 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,451 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,451 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-3-shard-people-testLeadershipTransferOnShutdown with journal batch size 1 09:32:24,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:24,452 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:32:24,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:24,452 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,452 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:24,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:24,452 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:24,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:24,452 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:24,452 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@5b0cc51b, attempting to connect 09:32:24,452 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@25aeb7b3, attempting to connect 09:32:24,454 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,454 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:32:24,454 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,454 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:24,454 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-2-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown#-1466745726] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-1556076539] 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-notifier#-58477607 created and ready for shard:member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-notifier#242442719 created and ready for shard:member-3-shard-people-testLeadershipTransferOnShutdown 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/shard-member-3:snapshot-read#-300897543] 09:32:24,455 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown#869839140] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 09:32:24,455 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testLeadershipTransferOnShutdown, matchIndex: -1, lastIndex: -1, isVoting: true 09:32:24,455 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=13800, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:24,455 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Follower's log matches - sending ElectionTimeout 09:32:24,455 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,455 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,456 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leader transfer complete 09:32:24,456 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 09:32:24,456 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testLeadershipTransferOnShutdown_member-1-shard-cars-testLeadershipTransferOnShutdown$a], minVersion=MAGNESIUM, maxVersion=CHLORINE_SR2}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: true. 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-people-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-3-shard-people-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=3, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-people-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [cars] 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-cars-testLeadershipTransferOnShutdown is initialized 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Initializing shard [people] 09:32:24,457 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-3-shard-people-testLeadershipTransferOnShutdown is initialized 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,457 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,457 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:24,457 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , received role change from null to Follower 09:32:24,458 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-people-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:24,458 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,458 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-people-testLeadershipTransferOnShutdown from null to Follower 09:32:24,458 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLeadershipTransferOnShutdown set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,458 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onLeaderTransitioning: LeaderTransitioning [leaderId=member-1-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,458 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:32:24,458 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testLeadershipTransferOnShutdown , registered listener akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown 09:32:24,458 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:32:24,458 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-3-shard-cars-testLeadershipTransferOnShutdown from null to Follower 09:32:24,460 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49be8d2b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7033ccfa 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7033ccfa 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@78006020 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50d5a01 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=750745150700} 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-847: slice: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE}} 09:32:24,460 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Received TimeoutNow - switching to Candidate 09:32:24,460 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,460 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-847: Message size: 390 bytes, total slices to send: 1 09:32:24,460 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-847: Message does not need to be sliced - sending original message 09:32:24,460 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Starting new election term 5 09:32:24,461 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@18f28c71 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2770a834 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@7bbfb603, splicing queue 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2770a834 queue spliced 0 messages 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageAssembler - member-2-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 854.2 μs 09:32:24,461 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Follower to Candidate 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:32:24,461 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Follower to Candidate 09:32:24,462 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:24,462 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:24,462 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:24,462 AM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Cluster state: [] 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 4 - updating term 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,462 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:32:24,464 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,464 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testLeadershipTransferOnShutdown: allocating new open transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: readying transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 ready 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: transitioned member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 to ready 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating canCommit 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 validated 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed canCommit 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Cluster state: [] 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=null 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer in progress - processing RequestVote 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testLeadershipTransferOnShutdown, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Set currentTerm=5, votedFor=member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,465 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:32:24,466 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:32:24,466 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageSlicer - member-1-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-851: Closing 09:32:24,466 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: false, hasLeader : false 09:32:24,466 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLeadershipTransferOnShutdown is no longer the leader 09:32:24,466 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,466 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLeadershipTransferOnShutdown , received role change from Leader to Follower 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Election: Leader has following peers: [member-3-shard-cars-testLeadershipTransferOnShutdown, member-1-shard-cars-testLeadershipTransferOnShutdown] 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,466 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=-1] 09:32:24,466 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-1-shard-cars-testLeadershipTransferOnShutdown from Leader to Follower 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLeadershipTransferOnShutdown: onLeadershipChange, isLeader: true, hasLeader : true 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: became leader with frontend state for [] 09:32:24,466 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,467 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,467 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 09:32:24,467 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=750745150700, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=0}} 09:32:24,467 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 09:32:24,467 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:24,467 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLeadershipTransferOnShutdown , received role change from Candidate to Leader 09:32:24,467 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received role changed for member-2-shard-cars-testLeadershipTransferOnShutdown from Candidate to Leader 09:32:24,467 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:32:24,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,467 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,467 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testLeadershipTransferOnShutdown, leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,467 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:24,467 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:24,467 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:32:24,467 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown: Closing 09:32:24,467 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-2-shard-cars-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:24,468 AM [cluster-test-shard-dispatcher-26] [INFO] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLeadershipTransferOnShutdown: Successfully transferred leadership to member-2-shard-cars-testLeadershipTransferOnShutdown in 16.41 ms 09:32:24,468 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: leader transfer succeeded - sending PoisonPill 09:32:24,468 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-1-shard-cars-testLeadershipTransferOnShutdown status sync done false 09:32:24,468 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.messages.ShardLeaderStateChanged] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown#-1279410782] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#1712073238] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown-notifier#1712073238] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,468 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,468 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,470 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:24,470 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-1-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:32:24,470 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:24,470 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,470 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,567 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Term 4 in "AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=3, leaderAddress=null, entries=[]]" message is greater than follower's term 0 - updating term 09:32:24,567 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Set currentTerm=4, votedFor=null 09:32:24,567 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown: Closing 09:32:24,567 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: getLeaderAddress leaderId = member-1-shard-people-testLeadershipTransferOnShutdown peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown 09:32:24,567 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-people-testLeadershipTransferOnShutdown, leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderPayloadVersion=12] 09:32:24,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:24,568 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:24,568 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All Shards are ready - data store testLeadershipTransferOnShutdown is ready 09:32:24,568 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done false 09:32:24,573 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: -1, nextIndex: 0 09:32:24,573 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:24,585 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-cars-testLeadershipTransferOnShutdown status sync done true 09:32:24,585 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:24,686 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown Received follower initial sync status for member-3-shard-people-testLeadershipTransferOnShutdown status sync done true 09:32:24,706 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-cars-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:25,115 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:32:25,474 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:25,474 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,474 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,474 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1464818602] 09:32:25,475 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:25,475 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7296e2c8] 09:32:25,475 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7296e2c8] 09:32:25,475 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@33f1bd31, attempting to connect 09:32:25,475 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 09:32:25,476 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=14450, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:25,476 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@747b2ddc instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} previous state SEALED 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=2, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=2, protocol=THREE_PHASE} 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 0 09:32:25,477 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@5a62fbf8 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=true}, enqueuedTicks=750744937285} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=OPEN} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], modifications=1} 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating canCommit 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 validated 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=SUCCESSOR} 09:32:25,478 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed canCommit 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3e4a2104 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@35c2abd7 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4c52cf5e, splicing queue 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@35c2abd7 queue spliced 0 messages 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.390 ms 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=true} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=true}, enqueuedTicks=750744937285, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 09:32:25,478 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 09:32:25,478 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@6b39d723 09:32:25,479 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:25,479 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@6b39d723 09:32:25,479 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,479 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751772526744} 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@26494c87 09:32:25,479 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating preCommit 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 1 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,479 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@3df20b61 09:32:25,479 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:25,479 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50d5a01 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751772774088} 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 candidate ready 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: canCommit - messages: [] 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 prepared 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:25,479 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed preCommit 09:32:25,480 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 09:32:25,480 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751772526744, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1}} 09:32:25,480 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:25,481 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating preCommit 09:32:25,481 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 candidate ready 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - candidate rootPath: / 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: canCommit - messages: [] 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 prepared 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:25,482 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 completed preCommit 09:32:25,483 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 09:32:25,483 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751772774088, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=1}} 09:32:25,483 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6697befb 09:32:25,483 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:25,483 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6697befb 09:32:25,483 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,483 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751777037321} 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@ad6fbda 09:32:25,483 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} allocated sequence 2 09:32:25,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 initiating commit 09:32:25,483 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@ad6fbda 09:32:25,483 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,483 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:25,483 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50d5a01 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751777226764} 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 170 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 submitted to persistence 09:32:25,484 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}]]] 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=170} 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:32:25,485 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 initiating commit 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,486 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=170}] 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,486 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 0 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 submitted to persistence 09:32:25,487 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:32:25,488 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-3-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=174} 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Log size is now 1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testLeadershipTransferOnShutdown, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7964ac0}, purged=MutableUnsignedLongSet{size=0}} 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751777037321, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=2}} 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 09:32:25,489 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1489109402] with name member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@747b2ddc completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7fceb12d 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:25,489 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1489109402] 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} allocated sequence 3 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@10e3fdd1 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751783195565} 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#-1489109402] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:25,490 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,490 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 86.89 μs 09:32:25,490 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,490 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:32:25,491 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,491 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 09:32:25,491 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 09:32:25,491 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:25,491 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 09:32:25,492 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=170} 09:32:25,492 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,492 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,493 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}]]] 09:32:25,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-3-shard-cars-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3d26fd3b}, purged=MutableUnsignedLongSet{size=0}} 09:32:25,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1090303511] with name member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher 09:32:25,493 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:32:25,493 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown/member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher#1090303511] 09:32:25,493 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,493 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 70.82 μs 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Setting last applied to 0 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendMetadata - member-1-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@480afb40}, purged=MutableUnsignedLongSet{size=0}} 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2: commit - successfulFromPrevious: [] 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 09:32:25,493 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#260596917] with name member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:32:25,494 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#260596917] 09:32:25,494 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,494 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-people-testLeadershipTransferOnShutdown updated: matchIndex: 0, nextIndex: 1 09:32:25,494 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,494 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:25,494 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:32:25,494 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-1-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 69.31 μs 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}]]] 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 2 09:32:25,494 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=1, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 09:32:25,495 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62}] 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-3-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@53f65e7b}, purged=MutableUnsignedLongSet{size=0}} 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1756775496] with name member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 09:32:25,495 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751777226764, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=2}} 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@49be8d2b completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:32:25,495 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4370003c 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2} 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=SEALED}: initiating purge 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} allocated sequence 3 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@6729636 09:32:25,496 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:25,496 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@73ff5c08 09:32:25,496 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@50d5a01 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751789262528} 09:32:25,496 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=false}, enqueuedTicks=751789455261} 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Commit index set to 0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Setting last applied to 0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Moving last applied index from -1 to 0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Applying state for log index 0 data Simple{size=174} 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-people-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendMetadata - member-2-shard-people-testLeadershipTransferOnShutdown: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@712f662}, purged=MutableUnsignedLongSet{size=0}} 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=2} 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testLeadershipTransferOnShutdown: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1105287527] with name member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 1, nextIndex: 2 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 1 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@1a837e7c 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,496 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new ready transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct canCommit 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Validating transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 validated 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct preCommit 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Preparing transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 candidate ready 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - candidate rootPath: / 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: canCommit - messages: [] 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 prepared 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 initiating direct commit 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Starting commit for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 229 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:25,497 AM [cluster-test-shard-dispatcher-23] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 submitted to persistence 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1105287527] 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751783195565, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=3}} 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE}: purge completed 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=DONE} 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 42.32 μs 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@747b2ddc purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1105287527] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:25,498 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown/member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher#-1756775496] 09:32:25,498 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-people-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,499 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-people-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-people-testLeadershipTransferOnShutdown-DTCL-publisher: 39.08 μs 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 0 to 1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, size=62} 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=1, prevLogTerm=5, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}]]] 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log Simple{size=229} 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 3 09:32:25,499 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,500 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2 09:32:25,500 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 09:32:25,500 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 09:32:25,500 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 2 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 2, nextIndex: 3 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 2 09:32:25,500 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=229}] 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 09:32:25,500 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=1 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Resuming commit of transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1: commit - successfulFromPrevious: [] 09:32:25,500 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:32:25,500 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=false} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 09:32:25,500 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505], coordinated=false}, enqueuedTicks=751789455261, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCommitSuccess{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=0}} 09:32:25,500 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED} directCommit completed 09:32:25,500 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=FLUSHED}: initiating purge 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@78eea3a2 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751794297594} 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 09:32:25,501 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-2-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 33.64 μs 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,501 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@148cf198: creating a new snapshot member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 09:32:25,501 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Replicate message: identifier: member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:25,501 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1 for transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-0 09:32:25,501 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3d918fc3: creating a new snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:25,501 AM [main] [DEBUG] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:25,501 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,501 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,501 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1307629676] 09:32:25,501 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} shard 1 09:32:25,502 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0 09:32:25,502 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:25,502 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@79027d61 09:32:25,502 AM [main] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:25,502 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}] 09:32:25,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:25,502 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:25,502 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown, primaryVersion=12] 09:32:25,502 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6a59109b, attempting to connect 09:32:25,502 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 2 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 2 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 1 to 2 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 2 data Simple{size=229} 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying foreign transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Committed transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,503 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testLeadershipTransferOnShutdown: publishChanges: rootPath: / 09:32:25,503 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=2 09:32:25,503 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testLeadershipTransferOnShutdown: Elapsed time for generation of change events for member-3-shard-cars-testLeadershipTransferOnShutdown-DTCL-publisher: 36.85 μs 09:32:25,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 3 is present for follower member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-cars-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 09:32:25,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} is not yet known 09:32:25,504 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: created state Enabled{clientId=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, nanosAgo=12170, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testLeadershipTransferOnShutdown, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}]]] 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Number of entries to be appended = 1 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Log size is now 4 09:32:25,505 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testLeadershipTransferOnShutdown, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:25,505 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@626721cc instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7c9a1946 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testLeadershipTransferOnShutdown updated: matchIndex: 3, nextIndex: 4 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Setting last applied to 3 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@7c9a1946 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: replication of member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@48526024 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 for transaction member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: finished purging transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@4d2c00c3 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@30c9997f transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=751795384972} 09:32:25,506 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@3d918fc3 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1cb5e609 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7ab4e4ae superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@57b3d923, splicing queue 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7ab4e4ae queue spliced 0 messages 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-3-frontend-datastore-testLeadershipTransferOnShutdown: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 539.0 μs 09:32:25,506 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63}] 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]}, enqueuedTicks=751794297594, sessionId=1, txSequence=5} with Envelope{sessionId=1, txSequence=5, message=TransactionPurgeResponse{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, sequence=1}} 09:32:25,506 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE}: purge completed 09:32:25,507 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, state=DONE} 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Commit index set to 3 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Setting last applied to 3 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Moving last applied index from 2 to 3 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1, size=63} 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, history=0, cookie=1} 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testLeadershipTransferOnShutdown: Purged transaction member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-1-1 09:32:25,507 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Persisting ApplyJournalEntries with index=3 09:32:25,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testLeadershipTransferOnShutdown: allocating new open snapshot member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1 09:32:25,507 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-848: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 09:32:25,508 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-848: Message size: 362 bytes, total slices to send: 1 09:32:25,508 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown_slicer-id-848: Message does not need to be sliced - sending original message 09:32:25,509 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 09:32:25,509 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1002395149], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=751795384972, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0}} 09:32:25,509 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=0} 09:32:25,509 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:32:25,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-3-shard-cars-testLeadershipTransferOnShutdown 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-3-shard-cars-testLeadershipTransferOnShutdown (Follower): Closing 09:32:25,509 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,509 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-3-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:32:25,509 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-3-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:25,509 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-3-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:25,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:25,510 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:25,510 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:32:25,510 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:32:25,510 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-3-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: onShutDown 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Initiating leader transfer 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@563674b2 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@563674b2 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testLeadershipTransferOnShutdown: active: false, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:25,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:32:25,511 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:32:25,511 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testLeadershipTransferOnShutdown (Follower): Closing 09:32:25,511 AM [cluster-test-shard-dispatcher-23] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:25,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@caa5fbf, splicing queue 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@32772e94 queue spliced 0 messages 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLeadershipTransferOnShutdown: refreshing backend for shard 1 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:25,511 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown#1464818602] 09:32:25,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:25,511 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testLeadershipTransferOnShutdown, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Leader, leaderId=null, activeMember=true] 09:32:25,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:25,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,646 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:25,648 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLeadershipTransferOnShutdown/member-1-shard-people-testLeadershipTransferOnShutdown#1843810145] to Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-people-testLeadershipTransferOnShutdown] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:25,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:25,765 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,765 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:25,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:25,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:25,885 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:25,885 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,4 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,4 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,364 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,364 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,604 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,604 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,604 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,604 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,625 AM [cluster-test-shard-dispatcher-23] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testLeadershipTransferOnShutdown: Failed to transfer leadership in 1.115 s 09:32:26,625 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: In unpauseLeader 09:32:26,625 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: leader transfer failed - sending PoisonPill 09:32:26,625 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader): Leadership transfer expired 09:32:26,626 AM [cluster-test-shard-dispatcher-23] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLeadershipTransferOnShutdown 09:32:26,626 AM [cluster-test-shard-dispatcher-23] [DEBUG] MessageSlicer - member-2-shard-cars-testLeadershipTransferOnShutdown (Leader)_slicer-id-855: Closing 09:32:26,626 AM [cluster-test-shard-dispatcher-23] [DEBUG] TestShard - member-2-shard-cars-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:26,626 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:32:26,626 AM [cluster-test-shard-dispatcher-23] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:26,627 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:26,627 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLeadershipTransferOnShutdown, generation=0} 09:32:26,627 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:32:26,627 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:26,627 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-cars-testLeadershipTransferOnShutdown 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Issuing gracefulStop to shard member-1-shard-people-testLeadershipTransferOnShutdown 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLeadershipTransferOnShutdown - waiting on 2 shards 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: onShutDown 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Initiating leader transfer 09:32:26,628 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testLeadershipTransferOnShutdown: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testLeadershipTransferOnShutdown, leaderId=null, leaderPayloadVersion=12] 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b667480 09:32:26,628 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:26,628 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardDataTree - member-1-shard-people-testLeadershipTransferOnShutdown: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@5b667480 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Attempting to transfer leadership 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-2-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Checking sendAppendEntries for follower member-3-shard-people-testLeadershipTransferOnShutdown: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:26,628 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,724 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,725 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,844 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,845 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,845 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,845 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:26,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:26,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:26,965 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,84 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,84 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,85 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,85 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,204 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,205 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,325 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,325 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,325 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,325 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,564 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,564 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,924 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:27,924 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:27,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:27,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,44 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,45 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,45 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,45 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,644 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,644 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,644 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,644 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:28,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:28,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:28,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,4 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,4 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,364 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,844 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:29,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:29,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:29,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,85 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,85 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,85 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,85 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,205 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,205 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,324 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,324 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,325 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,325 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,445 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,445 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,445 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,445 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,565 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,565 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,565 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,565 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,684 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,684 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,684 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,684 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,804 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:30,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:30,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:30,924 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,44 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,44 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,44 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,44 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,164 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,164 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,164 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,164 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,285 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,405 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,524 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,525 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,645 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,764 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:31,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:31,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:31,884 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,5 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,125 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,244 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,245 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,365 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,484 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,485 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,605 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,724 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,844 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,845 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:32,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:32,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:32,965 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,87 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,88 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,88 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,88 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,204 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,324 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,444 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,445 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,564 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,564 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,565 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,805 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:33,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:33,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:33,925 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,44 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,44 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,44 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,44 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,164 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,164 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,165 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,284 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,285 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,285 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,285 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,405 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,405 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,405 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,405 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,525 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,525 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,525 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,525 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,645 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,645 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,645 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,645 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,764 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:34,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:34,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:34,884 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,4 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:35,4 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,5 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:35,5 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,125 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:35,125 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,125 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:35,125 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,244 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:35,244 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,244 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:35,244 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,364 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:35,364 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,364 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:35,364 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,484 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testLeadershipTransferOnShutdown 09:32:35,484 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-2-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:35,484 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): sendAppendEntries: 1 is present for follower member-3-shard-people-testLeadershipTransferOnShutdown 09:32:35,484 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Sending AppendEntries to follower member-3-shard-people-testLeadershipTransferOnShutdown: AppendEntries [leaderId=member-1-shard-people-testLeadershipTransferOnShutdown, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-2, size=62}]]] 09:32:36,684 AM [cluster-test-shard-dispatcher-26] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testLeadershipTransferOnShutdown: Failed to transfer leadership in 10.06 s 09:32:36,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: In unpauseLeader 09:32:36,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: leader transfer failed - sending PoisonPill 09:32:36,684 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown (Leader): Leadership transfer expired 09:32:36,685 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-1-shard-people-testLeadershipTransferOnShutdown 09:32:36,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-people-testLeadershipTransferOnShutdown (Leader)_slicer-id-850: Closing 09:32:36,685 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-1-shard-people-testLeadershipTransferOnShutdown: Aborting 0 pending queued transactions 09:32:36,685 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardManager - shard-manager-testLeadershipTransferOnShutdown: All shards shutdown - sending PoisonPill to self 09:32:36,685 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - Stopping ShardManager shard-manager-testLeadershipTransferOnShutdown 09:32:36,686 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:36,686 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLeadershipTransferOnShutdown: shutting down 09:32:36,686 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLeadershipTransferOnShutdown: Closing 09:32:36,686 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:36,686 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:36,692 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:36,692 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:36,692 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:36,692 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:36,699 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:36,707 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:36,707 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:37,703 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7789291332624578442] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:37,723 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [770728117965576881] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:37,727 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:37,727 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:37,728 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:37.729] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:37,758 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:37,758 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:37,760 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:37.761] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:37,788 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:37,794 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-7024462890029493943] 09:32:37,794 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:37,805 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:37,805 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:37,805 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:37,805 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:37,808 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:37,808 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:37,808 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:37,810 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:37,814 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4878481436632633956] 09:32:37,814 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:37,819 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:37,820 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:37,820 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:37,820 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:37,836 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:37,841 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8269808473583022069] 09:32:37,841 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:37,843 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:37,843 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:37,844 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:37,844 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:37,854 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:37,855 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:37,856 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,859 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,860 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,861 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 09:32:37,861 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:37,862 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5d463ca4 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4d79c581 09:32:37,862 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@566db245 09:32:37,862 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1714037557862, None)) 09:32:37,863 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037557861,0,0)) 09:32:37,863 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6379ed22 09:32:37,863 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4d79c581 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@35f4ac41 09:32:37,863 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 09:32:37,864 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: updated peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry on shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:32:37,864 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:32:37,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,864 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,864 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-509643710] 09:32:37,864 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:32:37,866 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:37,867 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1016288411] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-509643710] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1131624132] 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-1:snapshot-read#-1131624132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-198647915 created and ready for shard:member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,872 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-198647915] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,873 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:37,873 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 09:32:37,873 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:37,873 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:32:37,873 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:37,873 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:37,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 09:32:37,873 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 09:32:37,875 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#-1016288411 created and ready for shard:member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,875 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:32:37,875 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,880 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:32:37,883 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:37,885 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:37,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:37,890 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:37,892 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,893 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: no initial generation override, starting from 0 09:32:37,893 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:37,893 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@72dcc4dd to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@13eacbd7 09:32:37,893 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@10e710eb 09:32:37,893 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, 0, 1714037557893, None)) 09:32:37,894 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037557892,0,0)) 09:32:37,894 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:37,894 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,894 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@13eacbd7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@49c01e2a 09:32:37,894 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Creating local shard: member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,894 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2d5a39c 09:32:37,894 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} started 09:32:37,895 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:37,895 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,895 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,895 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:32:37,895 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, persistent : true 09:32:37,895 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1225440127 created and ready for shard:member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1554700549] 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-notifier#1225440127] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-52315758] 09:32:37,896 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with journal batch size 1 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-1554700549] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,896 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/shard-member-2:snapshot-read#-52315758] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,897 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1758768103 created and ready for shard:member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,897 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-notifier#-1758768103] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:37,897 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:37,897 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:37,897 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [people] 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is initialized 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Initializing shard [cars] 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is initialized 09:32:37,897 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:37,897 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,897 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,898 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:37,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:37,899 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:37,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from null to Follower 09:32:37,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , registered listener akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry 09:32:37,900 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,900 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,900 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from null to Follower 09:32:37,900 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - Peer address for peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:37,945 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:37,945 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:37,945 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:38,153 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:32:38,153 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received ElectionTimeout - switching to Candidate 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 1 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:38,172 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:32:38,172 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:32:38,419 AM [cluster-test-shard-dispatcher-34] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:38,432 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:38,432 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,432 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 09:32:38,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:38,462 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=2, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,462 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 2 09:32:38,509 AM [cluster-test-shard-dispatcher-34] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:38,692 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:38,692 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,692 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 09:32:38,742 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:38,742 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,743 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 3 09:32:38,813 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:38,813 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,814 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Peer address for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,823 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:38,823 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,823 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,823 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,823 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-3-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,824 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:38,824 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:38,824 AM [cluster-test-shard-dispatcher-34] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,824 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry to akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:38,854 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:39,2 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:39,2 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,2 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 09:32:39,2 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,3 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:39,4 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:32:39,5 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 09:32:39,5 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:32:39,5 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,5 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,5 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:39,5 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:39,5 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 09:32:39,6 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 09:32:39,6 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:39,52 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Received ElectionTimeout 09:32:39,52 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,52 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 4 09:32:39,52 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,53 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:39,53 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:32:39,53 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:39,53 AM [cluster-test-shard-dispatcher-34] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:39,54 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=null 09:32:39,54 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:39,54 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,54 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:39,55 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [] 09:32:39,56 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:32:39,56 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:39,59 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:32:39,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: -1, nextIndex: 0 09:32:39,59 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:39,123 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 09:32:39,150 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:39,150 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 09:32:39,151 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] 09:32:39,151 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:39,151 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,151 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,151 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,151 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,151 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,151 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,151 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,151 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:39,152 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:39,152 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:39,152 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:39,152 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,152 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 09:32:39,152 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:39,152 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:39,152 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 09:32:39,152 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:39,152 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:39,152 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,152 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 09:32:39,152 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:39,152 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:39,152 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:39,152 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@4e3bd6a5 09:32:39,152 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:39,152 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@64b7c692 09:32:39,152 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:32:39,152 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:39,153 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,153 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:39,153 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,153 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,153 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,153 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@2a43eb93, attempting to connect 09:32:39,153 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4610b58a, attempting to connect 09:32:39,154 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:39,154 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=12360, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:39,154 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:39,154 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=14200, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:39,155 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:39,155 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@244106c6 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@172f1137 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765446129395} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 395 bytes, total slices to send: 1 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,156 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@43867132 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7c9548e3 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@16c15eb9, splicing queue 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7c9548e3 queue spliced 0 messages 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.031 ms 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13672234 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@e7d82fe 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765446219596} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 400 bytes, total slices to send: 1 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5ab8ff04 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b9262a1 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@61447d8e, splicing queue 09:32:39,157 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7b9262a1 queue spliced 0 messages 09:32:39,158 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 785.0 μs 09:32:39,158 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,158 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,158 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 ready 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 to ready 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating canCommit 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 ready 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 validated 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 to ready 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating canCommit 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 validated 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:39,159 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed canCommit 09:32:39,159 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed canCommit 09:32:39,159 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 09:32:39,159 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765446219596, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 09:32:39,159 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 09:32:39,159 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 09:32:39,159 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765446129395, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 09:32:39,160 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@128627e6 09:32:39,160 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@21845ed 09:32:39,160 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,160 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765453438539} 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@58c1806a 09:32:39,160 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 1 09:32:39,160 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@4fc83b7e 09:32:39,160 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,160 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765453557431} 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating preCommit 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating preCommit 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 candidate ready 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 candidate ready 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - candidate rootPath: / 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: canCommit - messages: [] 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: canCommit - messages: [] 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 prepared 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 prepared 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:39,160 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed preCommit 09:32:39,160 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed preCommit 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765453438539, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1}} 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765453557431, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=1}} 09:32:39,161 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@2ca59b2c 09:32:39,161 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@2ca59b2c 09:32:39,161 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,161 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765455149939} 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@78b38c02 09:32:39,161 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} allocated sequence 2 09:32:39,161 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@78b38c02 09:32:39,162 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,162 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765455303451} 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 initiating commit 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 initiating commit 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,163 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 submitted to persistence 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 submitted to persistence 09:32:39,163 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:39,164 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:39,164 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 09:32:39,164 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:39,164 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:39,164 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=184} 09:32:39,164 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:39,165 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 09:32:39,165 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=180} 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 1 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:39,165 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:39,165 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:39,165 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 0, nextIndex: 1 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 0 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendMetadata - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@28ebc912}, purged=MutableUnsignedLongSet{size=0}} 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3e078bbd}, purged=MutableUnsignedLongSet{size=0}} 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2: commit - successfulFromPrevious: [] 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:39,165 AM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:39,165 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822454958] with name member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:32:39,166 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2065078448] with name member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:32:39,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2065078448] 09:32:39,166 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:39,166 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:32:39,166 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822454958] 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 50.64 μs 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 30.99 μs 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-2065078448] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#822454958] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765455303451, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=2}} 09:32:39,166 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 09:32:39,166 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 0 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13672234 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 0 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from -1 to 0 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=180} 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 0 data Simple{size=184} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7c1faabe 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SEALED}: initiating purge 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} allocated sequence 3 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@61423ae6 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765460335316} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765455149939, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=2}} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@244106c6 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@77c2ebde}, purged=MutableUnsignedLongSet{size=0}} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@71fe4b5}, purged=MutableUnsignedLongSet{size=0}} 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4b476fa3 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} allocated sequence 3 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4a8c762f 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1901963673] with name member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1838236417] with name member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765460664863} 09:32:39,167 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:32:39,167 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=0 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1838236417] 09:32:39,167 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 33.56 μs 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#-1838236417] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1901963673] 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 32.88 μs 09:32:39,167 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher#1901963673] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:39,168 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,168 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,168 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,168 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 09:32:39,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 09:32:39,168 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 09:32:39,168 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:39,168 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 09:32:39,168 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:39,169 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 09:32:39,169 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 09:32:39,219 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,269 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,293 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}]]] 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:39,294 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72}] 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@3fca9dc8 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,295 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765460664863, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=3}} 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, size=72} 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE}: purge completed 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=DONE} 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:32:39,296 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@244106c6 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:32:39,296 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:32:39,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,320 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 09:32:39,320 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865] 09:32:39,320 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865] 09:32:39,362 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,362 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}]]] 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:39,364 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:39,365 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72}] 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 1 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@51b11869 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:32:39,365 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,366 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:32:39,366 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 1 09:32:39,366 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 1 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 0 to 1 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, size=72} 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765460335316, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=3}} 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE}: purge completed 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=DONE} 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 09:32:39,367 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13672234 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 0 for term 4 09:32:39,367 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=1 09:32:39,370 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865] 09:32:39,370 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:32:39,371 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:32:39,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:39,371 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:32:39,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} allocated sequence 0 09:32:39,371 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-0 09:32:39,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@6edb2ddf 09:32:39,371 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,371 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765664792523} 09:32:39,371 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,371 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 406 bytes, total slices to send: 1 09:32:39,371 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,371 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@21ad2c65 09:32:39,371 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,371 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765665210671} 09:32:39,371 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,372 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 402 bytes, total slices to send: 1 09:32:39,372 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 ready 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 to ready 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: readying transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:39,373 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 ready 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: transitioned member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 to ready 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:39,373 AM [cluster-test-shard-dispatcher-14] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765665210671, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} canCommit completed 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765664792523, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 09:32:39,374 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} canCommit completed 09:32:39,374 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:32:39,374 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:32:39,374 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} allocated sequence 0 09:32:39,374 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:32:39,374 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} allocated sequence 0 09:32:39,374 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-0 09:32:39,374 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:39,374 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,375 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,375 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,375 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,375 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,376 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,376 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 09:32:39,376 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 09:32:39,376 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 0 09:32:39,376 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@20d7cb28 09:32:39,376 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,376 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}, enqueuedTicks=765669760898} 09:32:39,376 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}} 09:32:39,376 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 490 bytes, total slices to send: 1 09:32:39,376 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,376 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 1 09:32:39,376 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@10ae2b1e 09:32:39,376 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}, enqueuedTicks=765670104524} 09:32:39,376 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, message: Envelope{sessionId=0, txSequence=6, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}} 09:32:39,376 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 491 bytes, total slices to send: 1 09:32:39,376 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,376 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} allocated sequence 2 09:32:39,376 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 09:32:39,377 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 09:32:39,377 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} allocated sequence 0 09:32:39,377 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e: creating a new transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 09:32:39,377 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 09:32:39,377 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} allocated sequence 0 09:32:39,377 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:39,377 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,377 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,377 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,378 AM [cluster-test-shard-dispatcher-14] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}, enqueuedTicks=765669760898, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0} 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2}, enqueuedTicks=765670104524, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 09:32:39,379 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1} 09:32:39,427 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,428 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,428 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,428 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,429 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config null, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-863: Closing 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-862: Closing 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: removing frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,430 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 09:32:39,430 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 09:32:39,430 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Leader to Follower 09:32:39,430 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry from Leader to Follower 09:32:39,479 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:39,479 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,480 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:39,480 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@7b07be45 09:32:39,480 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,480 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765773540443} 09:32:39,480 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,480 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 452 bytes, total slices to send: 1 09:32:39,480 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,480 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@663f7a88 09:32:39,480 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,480 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=765774041042} 09:32:39,480 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: slice: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} 09:32:39,480 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message size: 435 bytes, total slices to send: 1 09:32:39,481 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry_slicer-id-859: Message does not need to be sliced - sending original message 09:32:39,481 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 09:32:39,481 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-0 09:32:39,481 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 09:32:39,481 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-0 09:32:39,481 AM [main] [DEBUG] ProxyHistory - Recorded skipped transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 09:32:39,481 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e readied transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-0 09:32:39,481 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionForwardedToLeaderAfterRetry 09:32:39,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:32:39,481 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Re-initializing to Follower with previous leaderId member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : true 09:32:39,482 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 09:32:39,482 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently active leader, rejecting request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false, paused: false 09:32:39,482 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,482 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: request Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1468499397], modifications=1, protocol=THREE_PHASE}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@55199cd4, splicing queue 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c984174 queue spliced 1 messages 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@38273bdb, splicing queue 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4812dab5 queue spliced 1 messages 09:32:39,489 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,490 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 09:32:39,490 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:39,490 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:39,490 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:39,490 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:39,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:39,490 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,491 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,491 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:39,491 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@751e598, attempting to connect 09:32:39,491 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@d13eadf, attempting to connect 09:32:39,493 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:39,493 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry$c], minVersion=MAGNESIUM, maxVersion=CHLORINE_SR2}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 09:32:39,493 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:39,493 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: not currently leader, rejecting request ConnectClientRequest{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/temp/_user_shardmanager-testTransactionForwardedToLeaderAfterRetry_member-1-shard-people-testTransactionForwardedToLeaderAfterRetry$d], minVersion=MAGNESIUM, maxVersion=CHLORINE_SR2}. isLeader: false, isLeaderActive: true,isLeadershipTransferInProgress: false. 09:32:39,494 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Connect attempt to cars failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,494 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,495 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,495 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Connect attempt to people failed, will retry org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:39,495 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 2 shard people failed, removing state java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,495 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,495 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,495 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 2, scheduling retry in 1 second java.util.concurrent.TimeoutException: Connection attempt failed at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.onConnectResponse(AbstractShardBackendResolver.java:168) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$connectShard$4(AbstractShardBackendResolver.java:161) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleConnectClient(Shard.java:602) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:372) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:39,531 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,531 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:39,532 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,532 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,532 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:32:39,532 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,533 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 09:32:39,534 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,535 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:32:39,535 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:39,535 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:32:39,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:39,535 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,535 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:39,535 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:39,535 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done false 09:32:39,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:32:39,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,536 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry status sync done true 09:32:39,582 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : people 09:32:39,582 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865] 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Received TimeoutNow - switching to Candidate 09:32:39,582 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} allocated sequence 1 09:32:39,582 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@4f987a3a 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Election: Candidate has following voting peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,582 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Starting new election term 5 09:32:39,582 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} allocated sequence 1 09:32:39,582 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=FLUSHED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@505cfbdf 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] to peer member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: false, hasLeader : false 09:32:39,582 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry is no longer the leader 09:32:39,583 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 skipping transactions [3, 4, 5] 09:32:39,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,583 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,583 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:39,583 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Follower to Candidate 09:32:39,583 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=-1] 09:32:39,583 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,583 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Follower to Candidate 09:32:39,583 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 09:32:39,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:39,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Cluster state: [] 09:32:39,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] with higher term appears reachable, updating term. 09:32:39,583 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:32:39,583 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=null 09:32:39,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, lastLogIndex=1, lastLogTerm=4] - currentTerm: 5, votedFor: null, lastIndex: 1, lastTerm: 4 09:32:39,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Set currentTerm=5, votedFor=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,584 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:32:39,584 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Election: Leader has following peers: [member-1-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: false, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeadershipChange, isLeader: true, hasLeader : true 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: became leader with frontend state for [member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry] 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,585 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry , received role change from Candidate to Leader 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received role changed for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry from Candidate to Leader 09:32:39,585 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:32:39,585 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 09:32:39,586 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:39,586 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: getLeaderAddress leaderId = member-2-shard-people-testTransactionForwardedToLeaderAfterRetry peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:39,586 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderPayloadVersion=12] 09:32:39,586 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:39,586 AM [cluster-test-shard-dispatcher-11] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:39,586 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All Shards are ready - data store testTransactionForwardedToLeaderAfterRetry is ready 09:32:39,586 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done false 09:32:39,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 1, nextIndex: 2 09:32:39,587 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:39,588 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry Received follower initial sync status for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry status sync done true 09:32:39,652 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 115, lastApplied : 1, commitIndex : 1 09:32:39,701 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 112, lastApplied : 1, commitIndex : 1 09:32:39,770 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 116, lastApplied : 1, commitIndex : 1 09:32:39,821 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 117, lastApplied : 1, commitIndex : 1 09:32:39,833 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:32:39,892 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 120, lastApplied : 1, commitIndex : 1 09:32:39,941 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 09:32:40,15 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 122, lastApplied : 1, commitIndex : 1 09:32:40,61 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 09:32:40,131 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 114, lastApplied : 1, commitIndex : 1 09:32:40,181 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 09:32:40,251 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 118, lastApplied : 1, commitIndex : 1 09:32:40,301 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 119, lastApplied : 1, commitIndex : 1 09:32:40,376 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 124, lastApplied : 1, commitIndex : 1 09:32:40,427 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 124, lastApplied : 1, commitIndex : 1 09:32:40,491 AM [cluster-test-shard-dispatcher-21] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 114, lastApplied : 1, commitIndex : 1 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:40,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:40,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@522aa365] 09:32:40,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:40,510 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@72900d64] 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@522aa365] 09:32:40,510 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@72900d64] 09:32:40,511 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1097f7ca, attempting to connect 09:32:40,511 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3208792e, attempting to connect 09:32:40,511 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:40,511 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:40,512 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:40,512 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} previous state FLUSHED 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} previous state SEALED 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} previous state SEALED 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} previous state SEALED 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=OPEN} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 0 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1288/0x0000000801352c78@3e7431f5 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765773540443} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 replayed 1 successful requests 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 1 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@4f987a3a 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765876088418} 09:32:40,513 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,513 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating canCommit 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=OPEN} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 validated 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 0 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@7b07be45 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed canCommit 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765773540443} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating preCommit 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=2} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 candidate ready 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - candidate rootPath: / 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 replayed 2 successful requests 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=OPEN} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: canCommit - messages: [] 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 prepared 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed preCommit 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=OPEN} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating canCommit 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 validated 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=OPEN} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1} 09:32:40,514 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed canCommit 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=SUCCESSOR} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=SUCCESSOR} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=SUCCESSOR} 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@3615f9eb 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6ece5837 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@445b4b52, splicing queue 09:32:40,514 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6ece5837 queue spliced 0 messages 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.101 ms 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3600a2e1 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} previous state FLUSHED 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} previous state FLUSHED 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3600a2e1 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Forwarding successful request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=OPEN} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 0 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1288/0x0000000801352c78@3e7431f5 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} scheduling timeout in 30 seconds 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765774041042} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 replayed 1 successful requests 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,515 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 1 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} enqueing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@505cfbdf 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765876429804} 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating canCommit 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3600a2e1 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 validated 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=OPEN} 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed canCommit 09:32:40,516 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], modifications=1, protocol=THREE_PHASE} 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating preCommit 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 candidate ready 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - candidate rootPath: / 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: canCommit - messages: [] 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: preCommit - successfulFromPrevious: [] 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 prepared 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,516 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed preCommit 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 0 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@663f7a88 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765774041042} 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4fd7647e finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 09:32:40,524 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2fe26d74 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1daa8759 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@4f7a470a, splicing queue 09:32:40,524 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1daa8759 queue spliced 0 messages 09:32:40,524 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating canCommit 09:32:40,524 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,525 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 validated 09:32:40,525 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} in 9.823 ms 09:32:40,525 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed canCommit 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765773540443, sessionId=2, txSequence=0} with Envelope{sessionId=2, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765876088418, sessionId=2, txSequence=1} with Envelope{sessionId=2, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=1}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765773540443, sessionId=2, txSequence=2} with Envelope{sessionId=2, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} canCommit completed 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765774041042, sessionId=3, txSequence=0} with Envelope{sessionId=3, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=765876429804, sessionId=3, txSequence=1} with Envelope{sessionId=3, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=1}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=true}, enqueuedTicks=765774041042, sessionId=3, txSequence=2} with Envelope{sessionId=3, txSequence=2, message=TransactionCanCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=0}} 09:32:40,525 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} canCommit completed 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 2 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1d361d0f 09:32:40,525 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,525 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,525 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} allocated sequence 2 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1d361d0f 09:32:40,525 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,525 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766819183099} 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 2 09:32:40,525 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1173fd39 09:32:40,526 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 initiating commit 09:32:40,526 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,526 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,526 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} allocated sequence 2 09:32:40,526 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1173fd39 09:32:40,526 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 78 09:32:40,526 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766819414293} 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 submitted to persistence 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 initiating commit 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 185 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 2 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 09:32:40,526 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 submitted to persistence 09:32:40,527 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 09:32:40,527 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}]]] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=78} 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}]]] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=185} 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=78}] 09:32:40,528 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=2, term=5, payload=Simple{size=185}] 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=2, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 105, lastApplied : 1, commitIndex : 1 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 2, nextIndex: 3 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 2 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1: commit - successfulFromPrevious: [] 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2: commit - successfulFromPrevious: [] 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 changing state from COMMIT_PENDING to COMMITTED 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:32:40,533 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 09:32:40,533 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766819183099, sessionId=2, txSequence=3} with Envelope{sessionId=2, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=2}} 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} doCommit completed 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 3 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 74.69 μs 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@376a130b 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 8.340 μs 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,533 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SEALED}: initiating purge 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} allocated sequence 3 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7e33caf0 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827111543} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766819414293, sessionId=3, txSequence=3} with Envelope{sessionId=3, txSequence=3, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=2}} 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} doCommit completed 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: initiating purge 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} allocated sequence 3 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@12747b11 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 1 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SEALED}: initiating purge 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} allocated sequence 3 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@6f4b5989 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@75e1d01f 09:32:40,534 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,534 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,534 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827601612} 09:32:40,534 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 1 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@6f4b5989 09:32:40,534 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827940588} 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 1 09:32:40,534 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@4238ef0a 09:32:40,534 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating preCommit 09:32:40,534 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,534 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 09:32:40,534 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,534 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 candidate ready 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - candidate rootPath: / 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: canCommit - messages: [] 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 prepared 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed preCommit 09:32:40,534 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,535 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,535 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,535 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 1 09:32:40,535 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@4238ef0a 09:32:40,535 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766828724371} 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating preCommit 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 candidate ready 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - candidate rootPath: / 09:32:40,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: canCommit - messages: [] 09:32:40,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: preCommit - successfulFromPrevious: [] 09:32:40,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=185} 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 prepared 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,535 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed preCommit 09:32:40,535 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 1, lastIndex(): 2 09:32:40,535 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 1 to 2 09:32:40,535 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 2 data Simple{size=78} 09:32:40,536 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,536 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,536 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,536 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 09:32:40,536 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,536 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 09:32:40,536 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,536 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 24.53 μs 09:32:40,536 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,536 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,536 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 1 for term 5 09:32:40,536 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,536 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=2 09:32:40,536 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 58.06 μs 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827940588, sessionId=2, txSequence=5} with Envelope{sessionId=2, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=1}} 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766828724371, sessionId=3, txSequence=5} with Envelope{sessionId=3, txSequence=5, message=TransactionPreCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=1}} 09:32:40,537 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 2 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@5909c729 09:32:40,537 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,537 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,537 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} allocated sequence 2 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@5909c729 09:32:40,537 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766830781756} 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 2 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6cbb4a90 09:32:40,537 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,537 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,537 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} allocated sequence 2 09:32:40,537 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6cbb4a90 09:32:40,537 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766830948869} 09:32:40,537 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,537 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}]]] 09:32:40,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}]]] 09:32:40,537 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 initiating commit 09:32:40,537 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 initiating commit 09:32:40,537 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 223 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,538 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 submitted to persistence 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 3 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,538 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 submitted to persistence 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}]]] 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=2, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}]]] 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,539 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=3, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,539 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 09:32:40,539 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 3, nextIndex: 4 09:32:40,539 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:32:40,540 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,540 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 4 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}]]] 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73}] 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=3, prevLogTerm=5, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}]]] 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,540 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:32:40,540 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=223} 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@5aec16a7 09:32:40,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 09:32:40,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 09:32:40,541 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=4, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=3, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73}] 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 3 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827111543, sessionId=2, txSequence=4} with Envelope{sessionId=2, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, sequence=3}} 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@16ddca27 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE}: purge completed 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=DONE} 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, state=SUCCESSOR} 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 2 for term 5 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=3 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766827601612, sessionId=3, txSequence=4} with Envelope{sessionId=3, txSequence=4, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, sequence=3}} 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE}: purge completed 09:32:40,541 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3600a2e1 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=DONE} 09:32:40,541 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR}: purge completed 09:32:40,541 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, state=SUCCESSOR} 09:32:40,541 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=237}] 09:32:40,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=4, term=5, payload=Simple{size=223}] 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 4, nextIndex: 5 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 4 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1: commit - successfulFromPrevious: [] 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2: commit - successfulFromPrevious: [] 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 changing state from COMMIT_PENDING to COMMITTED 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766830781756, sessionId=2, txSequence=6} with Envelope{sessionId=2, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=2}} 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} doCommit completed 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 45.04 μs 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: initiating purge 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} allocated sequence 3 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@22db6885 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 23.31 μs 09:32:40,542 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SEALED}: initiating purge 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} allocated sequence 3 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@232376e6 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766835890502} 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766830948869, sessionId=3, txSequence=6} with Envelope{sessionId=3, txSequence=6, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=2}} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} doCommit completed 09:32:40,542 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 09:32:40,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: initiating purge 09:32:40,543 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} allocated sequence 0 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} allocated sequence 3 09:32:40,543 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@473e0b9c 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@5f60289a 09:32:40,543 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766836372031} 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-people-testTransactionForwardedToLeaderAfterRetry#-36871041], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SEALED}: initiating purge 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} allocated sequence 3 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@2606fcb3 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct canCommit 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,543 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766836310170} 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 validated 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct preCommit 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,543 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,543 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 09:32:40,543 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 candidate ready 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - candidate rootPath: / 09:32:40,543 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: canCommit - messages: [] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: preCommit - successfulFromPrevious: [] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 prepared 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 initiating direct commit 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 325 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 submitted to persistence 09:32:40,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 09:32:40,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 09:32:40,546 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2, size=73} 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-2 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=223} 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,547 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 09:32:40,547 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 37.61 μs 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 2, lastIndex(): 4 09:32:40,547 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 3 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 2 to 3 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1, size=73} 09:32:40,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-1-1 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 4 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 3 to 4 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 4 data Simple{size=237} 09:32:40,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,548 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,548 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 3 for term 5 09:32:40,548 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 32.87 μs 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}], SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}]]] 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=325} 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 09:32:40,548 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=6, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 5 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 6, nextIndex: 7 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@2035a944 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 6 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1: commit - successfulFromPrevious: [] 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 09:32:40,549 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766835890502, sessionId=2, txSequence=7} with Envelope{sessionId=2, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, sequence=3}} 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE}: purge completed 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=DONE} 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR}: purge completed 09:32:40,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5cdcbd2d purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, state=SUCCESSOR} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73}] 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=6, term=5, payload=Simple{size=325}] 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=4 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=4, prevLogTerm=5, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}]]] 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, logLastIndex=5, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 6 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1, size=73} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-1 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 6 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 5 to 6 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 6 data Simple{size=325} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,550 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 5 for term 5 09:32:40,551 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=5, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73}] 09:32:40,551 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=6 09:32:40,551 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,551 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 77.52 μs 09:32:40,551 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,551 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 72.12 μs 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766836372031, sessionId=2, txSequence=8} with Envelope{sessionId=2, txSequence=8, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=0}} 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED} directCommit completed 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=FLUSHED}: initiating purge 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} allocated sequence 1 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@3d1b75e9 09:32:40,552 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} allocated sequence 0 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,552 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@7ae70cd6 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766845461935} 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-people-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 5, nextIndex: 6 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 5, lastAppliedIndex: 4 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 5 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@1854ca55 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766836310170, sessionId=3, txSequence=7} with Envelope{sessionId=3, txSequence=7, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, sequence=3}} 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE}: purge completed 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@3600a2e1 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=DONE} 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR}: purge completed 09:32:40,552 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4451a8b2 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, state=SUCCESSOR} 09:32:40,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 09:32:40,553 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766845559387} 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct canCommit 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 validated 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct preCommit 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 candidate ready 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - candidate rootPath: / 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: canCommit - messages: [] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: preCommit - successfulFromPrevious: [] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 prepared 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 initiating direct commit 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 7, leaderLastIndex: 8, leaderSnapShotIndex: 5 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 7 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 09:32:40,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 submitted to persistence 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 5 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 5, context.getLastApplied(): 4, lastIndex(): 5 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 5 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 4 to 5 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 5 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2, size=73} 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-2-2 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 4 for term 5 09:32:40,554 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=5 09:32:40,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=6, prevLogTerm=5, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}], SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}]]] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=8, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=7, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73}] 09:32:40,555 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=8, term=5, payload=Simple{size=237}] 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 8, nextIndex: 9 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 8, lastAppliedIndex: 6 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 7 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@4316991f 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766845461935, sessionId=2, txSequence=9} with Envelope{sessionId=2, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, sequence=1}} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 8 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE}: purge completed 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, state=DONE} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1: commit - successfulFromPrevious: [] 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 changing state from COMMIT_PENDING to COMMITTED 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766845559387, sessionId=2, txSequence=10} with Envelope{sessionId=2, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=0}} 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED} directCommit completed 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=FLUSHED}: initiating purge 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} allocated sequence 1 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@285cd2fc 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 46.03 μs 09:32:40,556 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766849872980} 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 8, leaderSnapShotIndex: 7 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 09:32:40,556 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,556 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} allocated sequence 0 09:32:40,556 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@32ee5d7f 09:32:40,556 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766850251957} 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 8 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 8, context.getLastApplied(): 6, lastIndex(): 8 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new ready transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 7 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 6 to 7 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1, size=73} 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct canCommit 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-3-1 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Validating transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 8 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 7 to 8 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 8 data Simple{size=237} 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 validated 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct preCommit 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Preparing transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 candidate ready 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - candidate rootPath: / 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: canCommit - messages: [] 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: preCommit - successfulFromPrevious: [] 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 prepared 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 initiating direct commit 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Starting commit for transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 237 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 7 for term 5 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 9, leaderLastIndex: 10, leaderSnapShotIndex: 7 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,557 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=8 09:32:40,557 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,557 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 37.43 μs 09:32:40,557 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 09:32:40,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 submitted to persistence 09:32:40,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 09:32:40,558 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=8, prevLogTerm=5, leaderCommit=8, replicatedToAllIndex=7, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}], SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}]]] 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 2 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 8, entries to be added from: 0 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log Simple{size=237} 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 3 09:32:40,558 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=10, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=9, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73}] 09:32:40,559 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=10, term=5, payload=Simple{size=237}] 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 9 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 10, nextIndex: 11 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 8 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 9 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@710d97f3 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766849872980, sessionId=2, txSequence=11} with Envelope{sessionId=2, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, sequence=1}} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 10 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE}: purge completed 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, state=DONE} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Resuming commit of transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1: commit - successfulFromPrevious: [] 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1468499397], coordinated=false}, enqueuedTicks=766850251957, sessionId=2, txSequence=12} with Envelope{sessionId=2, txSequence=c, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=0}} 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED} directCommit completed 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=FLUSHED}: initiating purge 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} allocated sequence 1 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 29.82 μs 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@1ad206c8 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766854792934} 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persist data SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 09:32:40,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Replicate message: identifier: member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:40,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:40,561 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:40,561 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:40,561 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry#-324885978] 09:32:40,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:40,562 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: findLocalShard : cars 09:32:40,562 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:40,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.Shard$2 to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102] 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 10 09:32:40,562 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7856548a: creating a new snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 8, lastIndex(): 10 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 9 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 8 to 9 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 9 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1, size=73} 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-4-1 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 10 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 9 to 10 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 10 data Simple{size=237} 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying foreign transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,562 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:40,562 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,562 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,562 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Committed transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,562 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 1 09:32:40,562 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:40,562 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:40,562 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@712f9a93 09:32:40,562 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 9 for term 5 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=10 09:32:40,562 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: publishChanges: rootPath: / 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:40,562 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Elapsed time for generation of change events for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry-DTCL-publisher: 44.43 μs 09:32:40,562 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:40,562 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 09:32:40,563 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:40,563 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@a490069, attempting to connect 09:32:40,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): sendAppendEntries: 11 is present for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Sending AppendEntries to follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 09:32:40,563 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:40,564 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=7190, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, prevLogIndex=10, prevLogTerm=5, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}]]] 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Number of entries to be appended = 1 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Log size is now 2 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=11, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:40,564 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: persist complete SimpleReplicatedLogEntry [index=11, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73}] 09:32:40,564 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:40,564 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:40,564 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4f66b7cc instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3bc1dcd2 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@3bc1dcd2 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@ab041c9 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6efc55dc transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=766856048476} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7856548a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@5bee239d 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7110ea94 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@79a64376, splicing queue 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7110ea94 queue spliced 0 messages 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 405.7 μs 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry updated: matchIndex: 11, nextIndex: 12 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Applying to log - commitIndex: 11, lastAppliedIndex: 10 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Setting last applied to 11 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: replication of member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@3547d156 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: finished purging transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]} with Envelope{sessionId=2, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1468499397]}, enqueuedTicks=766854792934, sessionId=2, txSequence=13} with Envelope{sessionId=2, txSequence=d, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, sequence=1}} 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE}: purge completed 09:32:40,565 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@79d5254e purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, state=DONE} 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 09:32:40,565 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 09:32:40,566 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1 09:32:40,566 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-860: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 09:32:40,566 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-860: Message size: 513 bytes, total slices to send: 1 09:32:40,566 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry_slicer-id-860: Message does not need to be sliced - sending original message 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Commit index set to 11 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 11, context.getLastApplied(): 10, lastIndex(): 11 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Setting last applied to 11 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Moving last applied index from 10 to 11 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1, size=73} 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=1} 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Purged transaction member-2-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-5-1 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: fakeSnapshot purging log to 10 for term 5 09:32:40,566 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Persisting ApplyJournalEntries with index=11 09:32:40,567 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 09:32:40,567 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=766856048476, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0}} 09:32:40,567 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=0} 09:32:40,567 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} 09:32:40,567 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,567 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:40,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:40,567 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=0} shard 2 09:32:40,567 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-0 09:32:40,567 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:40,567 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:40,568 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@187df0ea 09:32:40,568 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:32:40,568 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, primaryVersion=12] 09:32:40,568 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@256df561, attempting to connect 09:32:40,569 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} is not yet known 09:32:40,569 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, nanosAgo=7340, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:40,569 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@13be8d09 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5fa09cc2 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} previous state OPEN 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5fa09cc2 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 for transaction member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} to successor RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=OPEN} enqueing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@62e4d25f 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:40,569 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@52f50ccb transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=766861329666} 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@7856548a finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, state=SUCCESSOR} 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@9e8dcdb 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@340849d7 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@70ba1323, splicing queue 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@340849d7 queue spliced 0 messages 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 373.7 μs 09:32:40,570 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: allocating new open snapshot member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-861: slice: identifier: member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-861: Message size: 360 bytes, total slices to send: 1 09:32:40,570 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry_slicer-id-861: Message does not need to be sliced - sending original message 09:32:40,571 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 09:32:40,571 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-741891523], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person}, enqueuedTicks=766861329666, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0}} 09:32:40,571 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people/person completed with ReadTransactionSuccess{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-2, sequence=0} 09:32:40,571 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Initiating leader transfer 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a308d58 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d24ee 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7a308d58 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7d24ee 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Attempting to transfer leadership 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:32:40,571 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:32:40,571 AM [cluster-test-shard-dispatcher-34] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:40,571 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@15e5e019, splicing queue 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6e4dff17 queue spliced 0 messages 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=2, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@580857b0, splicing queue 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2dcfe9b7 queue spliced 0 messages 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present}} 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=3, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=present} 09:32:40,572 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,572 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] 09:32:40,572 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Leader, leaderId=null, activeMember=true] 09:32:40,572 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:40,572 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Leader, leaderId=null, activeMember=true] 09:32:40,576 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-people-testTransactionForwardedToLeaderAfterRetry] 09:32:40,576 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onLeaderTransitioning: LeaderTransitioning [leaderId=member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry] 09:32:40,576 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:32:40,576 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:40,576 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderId=null, leaderPayloadVersion=12] 09:32:40,576 AM [cluster-test-shard-dispatcher-14] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:40,576 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClientActorBehavior - BackendInfo for shard 2 is now stale 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@52f50ccb superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4a672fc5, splicing queue 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@52f50ccb queue spliced 0 messages 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 2 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-people-testTransactionForwardedToLeaderAfterRetry#-40657865], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6efc55dc superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@7d85a815, splicing queue 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6efc55dc queue spliced 0 messages 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: refreshing backend for shard 1 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:40,577 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-127625222] 09:32:40,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:40,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 09:32:40,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:40,577 AM [cluster-test-shard-dispatcher-14] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Scheduling 2000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=null, activeMember=true] 09:32:40,577 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, matchIndex: 11, lastIndex: 11, isVoting: true 09:32:40,577 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Follower's log matches - sending ElectionTimeout 09:32:40,577 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 12, leaderLastIndex: 11, leaderSnapShotIndex: 10 09:32:40,578 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader): Leader transfer complete 09:32:40,578 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): tryToCompleteLeadershipTransfer: followerId: member-1-shard-people-testTransactionForwardedToLeaderAfterRetry, matchIndex: 5, lastIndex: 5, isVoting: true 09:32:40,578 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Follower's log matches - sending ElectionTimeout 09:32:40,578 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Checking sendAppendEntries for follower member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: active: true, followerNextIndex: 6, leaderLastIndex: 5, leaderSnapShotIndex: 4 09:32:40,578 AM [cluster-test-shard-dispatcher-34] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader): Leader transfer complete 09:32:40,578 AM [cluster-test-shard-dispatcher-11] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:40,578 AM [cluster-test-shard-dispatcher-14] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:40,731 AM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=5, success=true, followerId=member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry, logLastIndex=11, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 152, lastApplied : 11, commitIndex : 11 09:32:40,793 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 09:32:40,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Received ShardNotInitializedTimeout message for shard cars 09:32:40,793 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: An error occurred attempting to shut down the shards akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102]] after [200 ms]. Message of type [org.opendaylight.controller.cluster.raft.client.messages.Shutdown]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply. 09:32:40,793 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Returning NoShardLeaderException for shard cars 09:32:40,793 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:40,793 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-864: Closing 09:32:40,794 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:40,794 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:40,794 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry (Leader)_slicer-id-865: Closing 09:32:40,794 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:32:40,794 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:32:40,794 AM [ForkJoinPool.commonPool-worker-2] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:32:40,794 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:40,795 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager$ShardNotInitializedTimeout] from Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] to Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionForwardedToLeaderAfterRetry#-1690467536] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:40,795 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:40,795 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0} 09:32:40,795 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: shutting down 09:32:40,795 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:40,795 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: Issuing gracefulStop to shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry - waiting on 2 shards 09:32:40,796 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: onShutDown 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry 09:32:40,796 AM [cluster-test-shard-dispatcher-14] [INFO] TestShard - Stopping Shard member-1-shard-people-testTransactionForwardedToLeaderAfterRetry 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:40,796 AM [cluster-test-shard-dispatcher-14] [DEBUG] MessageAssembler - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry (Follower): Closing 09:32:40,796 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:32:40,796 AM [cluster-test-shard-dispatcher-14] [DEBUG] TestShard - member-1-shard-people-testTransactionForwardedToLeaderAfterRetry: Aborting 0 pending queued transactions 09:32:40,796 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionForwardedToLeaderAfterRetry: All shards shutdown - sending PoisonPill to self 09:32:40,797 AM [cluster-test-shard-dispatcher-14] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionForwardedToLeaderAfterRetry 09:32:40,797 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:40,797 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: Closing 09:32:40,798 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:40,798 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:40,802 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:40,802 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:40,802 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:40,802 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:40,814 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:40,814 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:41,814 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-7024462890029493943] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:41,831 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:41,832 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:41,833 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:41,834 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4878481436632633956] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [04/25/2024 09:32:41.834] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:41,841 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:41,841 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:41,843 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:41.844] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:41,863 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:41,869 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4602300612834893549] 09:32:41,870 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:41,871 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:41,871 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:41,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:41,877 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:41,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:41,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:41,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:41,892 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:41,897 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-6200029832265888754] 09:32:41,897 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:41,899 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:41,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:41,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:41,899 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:41,921 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:41,930 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:41,932 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-5743023735292763874] 09:32:41,932 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:41,933 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:41,935 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:41,935 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:41,935 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:41,935 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:41,942 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:41,943 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:41,945 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,954 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1616489646] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,954 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,957 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@25f1132d to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7585909f 09:32:41,957 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@32897540 09:32:41,957 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:41,957 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037561957, None)) 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037561956,0,0)) 09:32:41,957 AM [cluster-test-akka.actor.default-dispatcher-21] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7585909f to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@9030cbe 09:32:41,958 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:32:41,958 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:41,960 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:41,961 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,962 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e5e3657 09:32:41,962 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:41,962 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,963 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,963 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,963 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343] 09:32:41,963 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:32:41,963 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,963 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1432453611 created and ready for shard:member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,963 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:32:41,963 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-232480078] 09:32:41,964 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#-1432453611] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,964 AM [cluster-test-akka.actor.default-dispatcher-21] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-1:snapshot-read#-232480078] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,965 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:41,965 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:41,965 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:32:41,965 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:32:41,965 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:32:41,965 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:41,965 AM [cluster-test-akka.actor.default-dispatcher-19] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:32:41,965 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,965 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:32:41,970 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,972 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,973 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:41,973 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:32:41,973 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:41,973 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@35edc254 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11476c7e 09:32:41,973 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@f0510f6 09:32:41,974 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037561974, None)) 09:32:41,974 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037561973,0,0)) 09:32:41,974 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,974 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@11476c7e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@56397eb1 09:32:41,974 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,974 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:32:41,974 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:41,974 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,974 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5da67887 09:32:41,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:41,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,975 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164] 09:32:41,975 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:32:41,975 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:32:41,976 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,976 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-965409364] 09:32:41,976 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#924380678 created and ready for shard:member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,976 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-2:snapshot-read#-965409364] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:41,977 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:32:41,977 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:32:41,977 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:32:41,977 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:41,978 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:32:41,979 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#924380678] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:42,25 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1616489646] 09:32:42,25 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: findLocalShard : cars 09:32:42,25 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343] 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 1 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:32:42,243 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:42,244 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 09:32:42,244 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 09:32:42,539 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:32:42,539 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,539 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 2 09:32:42,580 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:42,839 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:32:42,839 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,839 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 3 09:32:42,889 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:42,889 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,890 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,898 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:42,963 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:43,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Received ElectionTimeout 09:32:43,143 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,143 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 4 09:32:43,144 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,144 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=null 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,145 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:43,145 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,145 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 09:32:43,146 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 09:32:43,146 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:32:43,146 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 09:32:43,146 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,147 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 09:32:43,147 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,182 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:43,184 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:43,186 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,197 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-667765226] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,198 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@5099a97e aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testLeadershipTransferOnShutdown-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1218731505]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@4539f43a 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLeadershipTransferOnShutdown: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:43,198 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@626721cc aborted transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1002395149], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@99be4a7 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1} 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1002395149], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@b21c7b4 09:32:43,198 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-3-frontend-datastore-testLeadershipTransferOnShutdown: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-3-frontend-datastore-testLeadershipTransferOnShutdown, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLeadershipTransferOnShutdown/member-2-shard-cars-testLeadershipTransferOnShutdown#1583019694], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,198 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@30c9997f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1002395149], modifications=0, protocol=ABORT}, enqueuedTicks=769492220344} 09:32:43,198 AM [Cleaner-0] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-852: slice: identifier: member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1002395149], modifications=0, protocol=ABORT}} 09:32:43,199 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-852: Message size: 282 bytes, total slices to send: 1 09:32:43,199 AM [Cleaner-0] [DEBUG] MessageSlicer - member-3-frontend-datastore-testLeadershipTransferOnShutdown_slicer-id-852: Message does not need to be sliced - sending original message 09:32:43,199 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-3-datastore-testLeadershipTransferOnShutdown-fe-0-txn-0-0} 09:32:43,199 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4f66b7cc aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-741891523], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@79776d8f 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1} 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-1-datastore-testTransactionForwardedToLeaderAfterRetry-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-741891523], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@7ca702a9 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,199 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionForwardedToLeaderAfterRetry, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionForwardedToLeaderAfterRetry/member-2-shard-cars-testTransactionForwardedToLeaderAfterRetry#-749495102], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:43,199 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: no initial generation override, starting from 0 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@5242155 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b52b56e 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@509bcd8c 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, 0, 1714037563200, None)) 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Creating local shard: member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:43,200 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,200 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037563199,0,0)) 09:32:43,201 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1b52b56e to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@38b75e69 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1dff0c1f 09:32:43,201 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} started 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21b03a55: creating a new snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1476036925] 09:32:43,201 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1476036925] 09:32:43,201 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx to akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,201 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: updated peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx on shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1476036925] 09:32:43,201 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} shard 1 09:32:43,201 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:43,201 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0 09:32:43,201 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:43,202 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@7a057838 09:32:43,202 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=12] 09:32:43,202 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} scheduling timeout in 2 seconds 09:32:43,202 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, persistent : true 09:32:43,202 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx with journal batch size 1 09:32:43,202 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, primaryVersion=12] 09:32:43,202 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@12503a5c, attempting to connect 09:32:43,203 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1476036925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,203 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1917314770 created and ready for shard:member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,203 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1987733182] 09:32:43,203 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-notifier#1917314770] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,203 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/shard-member-3:snapshot-read#1987733182] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,203 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:43,203 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:43,203 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initializing shard [cars] 09:32:43,203 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized 09:32:43,203 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:32:43,203 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:43,204 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from null to Follower 09:32:43,204 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,204 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,204 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , registered listener akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,204 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 09:32:43,204 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from null to Follower 09:32:43,204 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=7090, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:43,205 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@47de929a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@46edfef8 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@46edfef8 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@3e7e176f 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53c438ca transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=769495319827} 09:32:43,205 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21b03a55 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7af70674 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6bf14ecc superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@257af9b2, splicing queue 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6bf14ecc queue spliced 0 messages 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 666.1 μs 09:32:43,206 AM [cluster-test-shard-dispatcher-17] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new open snapshot member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-867: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-867: Message size: 245 bytes, total slices to send: 1 09:32:43,206 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-867: Message does not need to be sliced - sending original message 09:32:43,207 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 09:32:43,207 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=769495319827, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0}} 09:32:43,207 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0} 09:32:43,207 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,208 AM [cluster-test-shard-dispatcher-17] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,208 AM [cluster-test-shard-dispatcher-17] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-870: Closing 09:32:43,208 AM [cluster-test-shard-dispatcher-17] [DEBUG] TestShard - member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:32:43,208 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,208 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:43,208 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:43,211 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:43,211 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:43,211 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:43,211 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:43,213 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:43,229 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,229 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:32:43,229 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21b03a55: creating a new transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:32:43,230 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:32:43,230 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:43,230 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21b03a55 readied transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-0 09:32:43,230 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@5341d3ec 09:32:43,230 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,230 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53c438ca transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=1, protocol=SIMPLE}, enqueuedTicks=769523473665} 09:32:43,230 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-868: slice: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=1, protocol=SIMPLE}} 09:32:43,230 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-868: Message size: 408 bytes, total slices to send: 1 09:32:43,230 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx_slicer-id-868: Message does not need to be sliced - sending original message 09:32:43,230 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,231 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:32:43,231 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Re-initializing to Follower with previous leaderId member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,231 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:32:43,231 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : true 09:32:43,231 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:43,789 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:32:43,789 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:32:43,789 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:32:43,789 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Received ElectionTimeout - switching to Candidate 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Starting new election term 5 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] to peer member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: false, hasLeader : false 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is no longer the leader 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,790 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Follower to Candidate 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=-1] 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,790 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Follower to Candidate 09:32:43,790 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:43,790 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53c438ca superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@578a87bf, splicing queue 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@53c438ca queue spliced 1 messages 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,791 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] 09:32:43,791 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:43,791 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Candidate, leaderId=null, activeMember=true] 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Cluster state: [] 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Candidate in requestVote:RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Term 5 in "RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=null 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 5, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Set currentTerm=5, votedFor=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,792 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): requestVote returning: RequestVoteReply [term=5, voteGranted=true] 09:32:43,792 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=true], my term is 5 09:32:43,792 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=true], current voteCount: 1 09:32:43,792 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Election: Leader has following peers: [member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Candidate) :- Switching from behavior Candidate to Leader, election term: 5 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onLeadershipChange, isLeader: true, hasLeader : true 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: became leader with frontend state for [] 09:32:43,793 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx , received role change from Candidate to Leader 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is initialized - notifying 1 OnShardInitialized callbacks 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received role changed for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx from Candidate to Leader 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready - notifying 1 OnShardInitialized callbacks 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@16551690] 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:32:43,793 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:43,793 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@16551690] 09:32:43,793 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4dca23a0, attempting to connect 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} is not yet known 09:32:43,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,793 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: getLeaderAddress leaderId = member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,793 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, nanosAgo=8250, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:43,793 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done false 09:32:43,793 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@46edfef8 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1d3d4ef3 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1d3d4ef3 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 09:32:43,794 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalProxyTransaction - Not replaying IncrementTransactionSequenceRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], snapshotOnly=true} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Incrementing sequence org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$IncrementSequence@3f454de4 to successor LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1 replayed 1 successful requests 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1d3d4ef3 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=OPEN} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=1, protocol=SIMPLE} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} allocated sequence 0 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@5341d3ec 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@79235a2 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], coordinated=false}, enqueuedTicks=769523473665} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@21b03a55 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@7ca9032b 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@69964b37 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@40affd8c, splicing queue 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@69964b37 queue spliced 0 messages 09:32:43,796 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 2.439 ms 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: -1, nextIndex: 0 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: allocating new ready transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct canCommit 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Validating transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 validated 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct preCommit 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Preparing transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 candidate ready 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - candidate rootPath: / 09:32:43,796 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderPayloadVersion=12] 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: canCommit - messages: [] 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 prepared 09:32:43,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,796 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:43,796 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 initiating direct commit 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Starting commit for transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,797 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All Shards are ready - data store testTransactionRetryWithInitialAskTimeoutExOnCreateTx is ready 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 192 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 submitted to persistence 09:32:43,797 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}]]] 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Number of entries to be appended = 1 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Append entry to log Simple{size=192} 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Log size is now 1 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): handleAppendEntries returning : AppendEntriesReply [term=5, success=true, followerId=member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, logLastIndex=0, logLastTerm=5, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=0, term=5, payload=Simple{size=192}] 09:32:43,800 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx Received follower initial sync status for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx status sync done true 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx updated: matchIndex: 0, nextIndex: 1 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Setting last applied to 0 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Moving last applied index from -1 to 0 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Applying state for log index 0 data Simple{size=192} 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Resuming commit of transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@765b7476}, purged=MutableUnsignedLongSet{size=0}} 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, history=0, cookie=1} 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Committed transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1: commit - successfulFromPrevious: [] 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:32:43,800 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1209449516] with name member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persisting ApplyJournalEntries with index=0 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265], coordinated=false}, enqueuedTicks=769523473665, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=0}} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@5686ff1e 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265]} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1209449516] 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=SEALED}: initiating purge 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@62020ebe 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@79235a2 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265]}, enqueuedTicks=770094485728} 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: publishChanges: rootPath: / 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: purging transaction member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Persist data SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 09:32:43,801 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:43,801 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Replicate message: identifier: member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Elapsed time for generation of change events for member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher: 52.03 μs 09:32:43,801 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,801 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 09:32:43,801 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 09:32:43,802 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: persist complete SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}] 09:32:43,801 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-DTCL-publisher#1209449516] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,802 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Follower): Closing 09:32:43,802 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:32:43,802 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 09:32:43,802 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,802 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,802 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:43,802 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,802 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 09:32:43,802 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-3-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:43,802 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Issuing gracefulStop to shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,802 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx - waiting on 1 shards 09:32:43,802 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-667765226] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-667765226] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-667765226] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-667765226] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: onShutDown 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Initiating leader transfer 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1571591795] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1571591795] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:43,803 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderId=null, leaderPayloadVersion=12] 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1cc69d43 09:32:43,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@1cc69d43 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Attempting to transfer leadership 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@79235a2 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@62f9499, splicing queue 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}]]] 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: active: false, followerNextIndex: -1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@79235a2 queue spliced 1 messages 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: refreshing backend for shard 1 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:43,803 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:43,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Scheduling 1000 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:32:43,909 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:43,910 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}]]] 09:32:44,30 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:44,30 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}]]] 09:32:44,150 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:44,150 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}]]] 09:32:44,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4602300612834893549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:44,223 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4602300612834893549] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:44,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): sendAppendEntries: 1 is present for follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:44,269 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Sending AppendEntries to follower member-3-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: AppendEntries [leaderId=member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, prevLogIndex=0, prevLogTerm=5, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=5, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-1-1, size=84}]]] 09:32:44,389 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Failed to transfer leadership in 586.4 ms 09:32:44,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: In unpauseLeader 09:32:44,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: leader transfer failed - sending PoisonPill 09:32:44,389 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader): Leadership transfer expired 09:32:44,390 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:44,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx (Leader)_slicer-id-873: Closing 09:32:44,390 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Aborting 0 pending queued transactions 09:32:44,390 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: All shards shutdown - sending PoisonPill to self 09:32:44,390 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx 09:32:44,390 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:44,390 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0} 09:32:44,390 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: shutting down 09:32:44,390 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:44,391 AM [main] [WARN] ActorUtils - ShardManager for testTransactionRetryWithInitialAskTimeoutExOnCreateTx data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:32:44,391 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: Closing 09:32:44,391 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:44,391 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#-2086935368] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:44,391 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1920087265] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1920087265] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:44,391 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:44,392 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:45,403 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-6200029832265888754] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:45,411 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:45,411 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:45.414] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:45,420 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:45,421 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:45,422 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:45.423] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:45,450 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:45,454 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-841707087459359789] 09:32:45,454 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:45,455 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:45,455 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:45,461 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:45,461 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:45,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:45,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:45,466 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:45,468 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:45,472 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4072885758058018932] 09:32:45,472 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:45,475 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:45,475 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:45,481 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:45,481 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:45,490 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:45,506 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-9145925221775550011] 09:32:45,507 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:45,513 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:45,513 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:45,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:45,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:45,519 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:45,520 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:45,521 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,524 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-267510054] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,525 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@72fd48cf to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@661c8e51 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6223064a 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1714037565526, None)) 09:32:45,526 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037565525,0,0)) 09:32:45,527 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@661c8e51 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@14a08f09 09:32:45,527 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 09:32:45,527 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:45,528 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:45,529 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,531 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 09:32:45,531 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@3e09f374 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728] 09:32:45,533 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,533 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1545011142 created and ready for shard:member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-13] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1681207040] 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#-1545011142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1681207040] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,534 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,534 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#1804225403 created and ready for shard:member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,534 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-notifier#1804225403] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,535 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2053379668] 09:32:45,535 AM [cluster-test-shard-dispatcher-17] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,535 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,535 AM [cluster-test-shard-dispatcher-17] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-1-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073] 09:32:45,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:45,535 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:45,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,535 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,536 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 09:32:45,536 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 09:32:45,536 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:32:45,537 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-195001874] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,538 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:32:45,538 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,538 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:32:45,538 AM [cluster-test-akka.actor.default-dispatcher-13] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#2053379668] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,539 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:45,544 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 09:32:45,546 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:45,546 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:45,546 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 09:32:45,546 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is initialized 09:32:45,546 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,546 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,546 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:32:45,546 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,547 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testReadWriteTransactionWithMultipleShards 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,552 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:32:45,552 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:45,552 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2ec01b5a to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@513760ab 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,552 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,552 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2519f626 09:32:45,553 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, 0, 1714037565552, None)) 09:32:45,553 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037565551,0,0)) 09:32:45,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@52a4bd7 09:32:45,553 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@513760ab to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@767fe45f 09:32:45,553 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} started 09:32:45,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:45,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,553 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,553 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards on shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-122575025] 09:32:45,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,554 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testReadWriteTransactionWithMultipleShards, persistent : true 09:32:45,554 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards#-122575025] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,554 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,554 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testReadWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards on shard member-2-shard-people-testReadWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#115868171] 09:32:45,554 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:32:45,555 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Shard created : member-2-shard-people-testReadWriteTransactionWithMultipleShards, persistent : true 09:32:45,555 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards#115868171] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,555 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Starting recovery for member-2-shard-people-testReadWriteTransactionWithMultipleShards with journal batch size 1 09:32:45,555 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1315991751] 09:32:45,555 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1315991751] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,555 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#435971394 created and ready for shard:member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-notifier#435971394] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:45,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [cars] 09:32:45,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is initialized 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1169563865 created and ready for shard:member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-notifier#1169563865] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testReadWriteTransactionWithMultipleShards from null to Follower 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Initializing shard [people] 09:32:45,556 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards is initialized 09:32:45,556 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Peer address for peer member-1-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , received role change from null to Follower 09:32:45,556 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testReadWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards 09:32:45,556 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testReadWriteTransactionWithMultipleShards from null to Follower 09:32:45,557 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1294118937] 09:32:45,557 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#-1294118937] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:45,561 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:45,562 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:45,603 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-267510054] 09:32:45,604 AM [cluster-test-shard-dispatcher-33] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : cars 09:32:45,604 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728] 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] MessageAssembler - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,787 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:32:45,787 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Follower to Candidate 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testReadWriteTransactionWithMultipleShards is no longer the leader 09:32:45,807 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:32:45,807 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Follower to Candidate 09:32:46,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,87 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:32:46,127 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,127 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,127 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:32:46,137 AM [cluster-test-shard-dispatcher-26] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:46,137 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:46,327 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,327 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,327 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:32:46,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,446 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:32:46,478 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:46,478 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-33] [INFO] TestShard - Peer address for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,478 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-3-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,491 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testReadWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,525 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:46,636 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,636 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,636 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:32:46,637 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,638 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testReadWriteTransactionWithMultipleShards] 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:32:46,639 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:46,639 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testReadWriteTransactionWithMultipleShards from Candidate to Leader 09:32:46,640 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Closing 09:32:46,640 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,640 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:32:46,640 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:46,640 AM [cluster-test-shard-dispatcher-26] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:46,640 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done false 09:32:46,640 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:32:46,640 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:46,687 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:32:46,687 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,687 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:32:46,687 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,688 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testReadWriteTransactionWithMultipleShards] 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:32:46,688 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:32:46,689 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testReadWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:32:46,689 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:32:46,689 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:46,689 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:46,689 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testReadWriteTransactionWithMultipleShards from Candidate to Leader 09:32:46,689 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 09:32:46,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Closing 09:32:46,689 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testReadWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,689 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:32:46,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:46,689 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:46,689 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All Shards are ready - data store testReadWriteTransactionWithMultipleShards is ready 09:32:46,689 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done false 09:32:46,690 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:32:46,690 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:46,719 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-267510054] 09:32:46,719 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: findLocalShard : people 09:32:46,719 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073] 09:32:46,719 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@730f53ef: creating a new transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:32:46,720 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:32:46,720 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:46,720 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:46,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-195001874] 09:32:46,720 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 09:32:46,720 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:32:46,720 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:46,720 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:32:46,720 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:32:46,720 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:32:46,720 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-195001874] 09:32:46,720 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 09:32:46,720 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:32:46,720 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:46,720 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@730f53ef readied transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:32:46,720 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@2e16e099 09:32:46,720 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:46,720 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@520d19bc 09:32:46,720 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:32:46,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:46,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:32:46,720 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:32:46,721 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:32:46,721 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:32:46,721 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards, primaryVersion=12] 09:32:46,721 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@3265a532, attempting to connect 09:32:46,721 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@338f1019, attempting to connect 09:32:46,722 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 09:32:46,722 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=17160, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:46,722 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} is not yet known 09:32:46,722 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, nanosAgo=13070, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:46,723 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:46,724 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@428feaa2 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@26b56c1a 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@26b56c1a 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@2785cbe3 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:46,724 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=773013960083} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message size: 395 bytes, total slices to send: 1 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message does not need to be sliced - sending original message 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@730f53ef finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@55ca88b8 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@967c6d8 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@593109a4, splicing queue 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@967c6d8 queue spliced 0 messages 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 866.3 μs 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@629356e7 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cebee5e 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cebee5e 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@a20faf1 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=773014084626} 09:32:46,725 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}} 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message size: 400 bytes, total slices to send: 1 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message does not need to be sliced - sending original message 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@730f53ef finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@78971e0e 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30c84f9f superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@9d5a3b2, splicing queue 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@30c84f9f queue spliced 0 messages 09:32:46,726 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] ClientActorBehavior - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 694.8 μs 09:32:46,726 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,726 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: transitioned member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 09:32:46,727 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:46,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=773013960083, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=773014084626, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:32:46,728 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@568e7dd1 09:32:46,728 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@4f854db1 09:32:46,728 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:46,728 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773021862398} 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@75ff7d04 09:32:46,728 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 09:32:46,728 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@55c51026 09:32:46,728 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:46,728 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773022046031} 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:46,732 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:46,732 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773021862398, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773022046031, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 09:32:46,733 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@150dacde 09:32:46,733 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@150dacde 09:32:46,733 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:46,733 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773026729540} 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1b14e426 09:32:46,733 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 09:32:46,733 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@1b14e426 09:32:46,733 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:46,733 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773026826902} 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,734 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:32:46,734 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 09:32:46,735 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:46,735 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}]]] 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=184} 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:32:46,735 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:46,735 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testReadWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:46,736 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=184}] 09:32:46,736 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:32:46,736 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testReadWriteTransactionWithMultipleShards status sync done true 09:32:46,736 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testReadWriteTransactionWithMultipleShards status sync done true 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testReadWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@190e1493}, purged=MutableUnsignedLongSet{size=0}} 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@410ab384}, purged=MutableUnsignedLongSet{size=0}} 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:46,736 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1275586102] with name member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:32:46,736 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1275586102] 09:32:46,736 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#79119858] with name member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:32:46,737 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:32:46,737 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:32:46,737 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:46,737 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 52.68 μs 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#79119858] 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-1275586102] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#79119858] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 24.15 μs 09:32:46,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:32:46,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:32:46,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 09:32:46,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:32:46,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:46,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:32:46,737 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773026826902, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 09:32:46,737 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 09:32:46,737 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@629356e7 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=184} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@20db324a 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTree - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@607c2f31 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-people-testReadWriteTransactionWithMultipleShards#993591073], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773031344629} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773026729540, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@428feaa2 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendMetadata - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@3e7251f1}, purged=MutableUnsignedLongSet{size=0}} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@36d6d9c0 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4fe6e432 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-1-shard-cars-testReadWriteTransactionWithMultipleShards#-1866896728], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-611204137] with name member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#1076505859]}, enqueuedTicks=773031575063} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-611204137] 09:32:46,738 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1b503730}, purged=MutableUnsignedLongSet{size=0}} 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 38.07 μs 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards/member-2-shard-cars-testReadWriteTransactionWithMultipleShards-DTCL-publisher#-611204137] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1034289224] with name member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1034289224] 09:32:46,738 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testReadWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher: 30.61 μs 09:32:46,738 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards/member-2-shard-people-testReadWriteTransactionWithMultipleShards-DTCL-publisher#1034289224] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:46,739 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@730f53ef: creating a new snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:32:46,739 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:32:46,739 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:32:46,739 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@563ef410 09:32:46,739 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=773032586561} 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:32:46,741 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 09:32:46,741 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 09:32:46,741 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:46,741 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1 09:32:46,741 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-875: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:32:46,741 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] SlicedMessageState - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-875: Message size: 336 bytes, total slices to send: 1 09:32:46,741 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards_slicer-id-875: Message does not need to be sliced - sending original message 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}] 09:32:46,741 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}] 09:32:46,742 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:32:46,742 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=773032586561, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:32:46,742 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 09:32:46,742 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:32:46,742 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 09:32:46,742 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@1ce099e5 09:32:46,742 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=773035662403} 09:32:46,742 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testReadWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-876: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-876: Message size: 341 bytes, total slices to send: 1 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards_slicer-id-876: Message does not need to be sliced - sending original message 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1076505859], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=773035662403, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:32:46,743 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 09:32:46,743 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:46,743 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 09:32:46,744 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 09:32:46,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 09:32:46,744 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:32:46,744 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-people-testReadWriteTransactionWithMultipleShards (Follower): Closing 09:32:46,744 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:32:46,744 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:32:46,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:32:46,744 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:32:46,744 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards, generation=0} 09:32:46,744 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 09:32:46,744 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:32:46,744 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,744 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testReadWriteTransactionWithMultipleShards - waiting on 2 shards 09:32:46,744 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-195001874] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testReadWriteTransactionWithMultipleShards#-195001874] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: onShutDown 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: onShutDown 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Initiating leader transfer 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@97aec4e 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3e7e44e0 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [INFO] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@97aec4e 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@3e7e44e0 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:32:46,745 AM [cluster-test-shard-dispatcher-33] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,745 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 09:32:46,745 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 09:32:46,877 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,877 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=72}]]] 09:32:46,927 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testReadWriteTransactionWithMultipleShards 09:32:46,928 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testReadWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testReadWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=72}]]] 09:32:46,997 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Failed to transfer leadership in 252.5 ms 09:32:46,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: In unpauseLeader 09:32:46,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:32:46,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:32:46,998 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testReadWriteTransactionWithMultipleShards 09:32:46,998 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-880: Closing 09:32:46,998 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:32:47,46 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Failed to transfer leadership in 301.9 ms 09:32:47,46 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: In unpauseLeader 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testReadWriteTransactionWithMultipleShards 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testReadWriteTransactionWithMultipleShards (Leader)_slicer-id-881: Closing 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testReadWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:32:47,47 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] ShardManager - shard-manager-testReadWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:32:47,47 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testReadWriteTransactionWithMultipleShards 09:32:47,47 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:32:47,47 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: shutting down 09:32:47,47 AM [cluster-test-akka.actor.default-dispatcher-12] [DEBUG] MessageAssembler - member-1-frontend-datastore-testReadWriteTransactionWithMultipleShards: Closing 09:32:47,48 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:47,48 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:47,52 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:47,53 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:47,53 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:47,53 AM [cluster-test-akka.actor.default-dispatcher-30] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:47,55 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:47,69 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:47,69 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:48,65 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-841707087459359789] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:48,85 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4072885758058018932] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:48,89 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:48,89 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:48,90 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:48.092] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:48,98 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:48,98 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:48,100 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:48.101] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:48,112 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:48,116 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2976832276966484022] 09:32:48,116 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:48,117 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:48,117 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:48,118 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:48,118 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:48,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:48,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:48,129 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:48,131 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:48,139 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [3604784638137504102] 09:32:48,139 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:48,140 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:48,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:48,144 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:48,144 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:48,154 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:48,159 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [7064310291996408186] 09:32:48,159 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:48,162 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:48,162 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:48,163 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:48,163 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:48,169 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:48,170 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:48,174 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 09:32:48,189 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 09:32:48,192 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1955113081] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,195 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:48,196 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:48,198 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 09:32:48,198 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 09:32:48,198 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4dcaa839 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43b6ca44 09:32:48,198 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@478956ea 09:32:48,199 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testLargeReadReplySlicing, 0, 1714037568198, None)) 09:32:48,199 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037568197,0,0)) 09:32:48,199 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@43b6ca44 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@59c606c8 09:32:48,199 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} started 09:32:48,200 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:48,200 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:48,201 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:48,201 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 09:32:48,201 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-1-shard-cars-testLargeReadReplySlicing 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7100af38 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testLargeReadReplySlicing 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,202 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-1-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] 09:32:48,202 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-1-shard-cars-testLargeReadReplySlicing, persistent : true 09:32:48,202 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testLargeReadReplySlicing 09:32:48,203 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-1-shard-cars-testLargeReadReplySlicing with journal batch size 1 09:32:48,203 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,203 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-629076896] 09:32:48,203 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-1798088140 created and ready for shard:member-1-shard-cars-testLargeReadReplySlicing 09:32:48,203 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/shard-member-1:snapshot-read#-629076896] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,203 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing-notifier#-1798088140] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,205 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:48,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:48,205 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:48,206 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 09:32:48,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 09:32:48,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 09:32:48,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testLargeReadReplySlicing is initialized 09:32:48,206 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 09:32:48,206 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 09:32:48,208 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from null to Follower 09:32:48,209 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testLargeReadReplySlicing 09:32:48,210 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testLargeReadReplySlicing 09:32:48,210 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Creating local shard: member-2-shard-cars-testLargeReadReplySlicing 09:32:48,210 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:48,210 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: no initial generation override, starting from 0 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@418d91d9 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d991c24 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2847cc5d 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testLargeReadReplySlicing, 0, 1714037568211, None)) 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037568210,0,0)) 09:32:48,211 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79668727 09:32:48,211 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d991c24 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@615635e 09:32:48,211 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} started 09:32:48,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:48,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testLargeReadReplySlicing 09:32:48,212 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:48,212 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: updated peer member-1-shard-cars-testLargeReadReplySlicing on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing on shard member-2-shard-cars-testLargeReadReplySlicing whose actor address is Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-2026445132] 09:32:48,212 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testLargeReadReplySlicing, persistent : true 09:32:48,212 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testLargeReadReplySlicing with journal batch size 1 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1315769426] 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing#-2026445132] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#2006636681 created and ready for shard:member-2-shard-cars-testLargeReadReplySlicing 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/shard-member-2:snapshot-read#1315769426] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing-notifier#2006636681] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:48,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Initializing shard [cars] 09:32:48,213 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testLargeReadReplySlicing is initialized 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testLargeReadReplySlicing is no longer the leader 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , received role change from null to Follower 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:48,213 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testLargeReadReplySlicing , registered listener akka://cluster-test/user/shardmanager-testLargeReadReplySlicing 09:32:48,213 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-2-shard-cars-testLargeReadReplySlicing from null to Follower 09:32:48,262 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1955113081] 09:32:48,262 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: findLocalShard : cars 09:32:48,262 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower): Received ElectionTimeout - switching to Candidate 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testLargeReadReplySlicing] 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=1, votedFor=member-1-shard-cars-testLargeReadReplySlicing 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 1 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing (Follower): Closing 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: false, hasLeader : false 09:32:48,460 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testLargeReadReplySlicing is no longer the leader 09:32:48,460 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Follower to Candidate 09:32:48,461 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Follower to Candidate 09:32:48,750 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 09:32:48,750 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=2, votedFor=member-1-shard-cars-testLargeReadReplySlicing 09:32:48,750 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 2 09:32:48,769 AM [cluster-test-shard-dispatcher-27] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:49,29 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 09:32:49,29 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=3, votedFor=member-1-shard-cars-testLargeReadReplySlicing 09:32:49,29 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 3 09:32:49,130 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:49,130 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:49,130 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - Peer address for peer member-2-shard-cars-testLargeReadReplySlicing set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testLargeReadReplySlicing/member-3-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:49,172 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testLargeReadReplySlicing with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing to akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing 09:32:49,173 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:49,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Received ElectionTimeout 09:32:49,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 09:32:49,309 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Starting new election term 4 09:32:49,309 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testLargeReadReplySlicing 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Cluster state: [] 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=null 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testLargeReadReplySlicing, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Set currentTerm=4, votedFor=member-1-shard-cars-testLargeReadReplySlicing 09:32:49,310 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Election: Leader has following peers: [member-2-shard-cars-testLargeReadReplySlicing] 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [INFO] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:49,311 AM [cluster-test-shard-dispatcher-19] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testLargeReadReplySlicing: onLeadershipChange, isLeader: true, hasLeader : true 09:32:49,312 AM [cluster-test-shard-dispatcher-19] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: became leader with frontend state for [] 09:32:49,312 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=12] 09:32:49,312 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:49,312 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:49,312 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testLargeReadReplySlicing , received role change from Candidate to Leader 09:32:49,312 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received role changed for member-1-shard-cars-testLargeReadReplySlicing from Candidate to Leader 09:32:49,312 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing: Closing 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: getLeaderAddress leaderId = member-1-shard-cars-testLargeReadReplySlicing peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing Received follower initial sync status for member-2-shard-cars-testLargeReadReplySlicing status sync done false 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testLargeReadReplySlicing, leaderId=member-1-shard-cars-testLargeReadReplySlicing, leaderPayloadVersion=12] 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:49,312 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: All Shards are ready - data store testLargeReadReplySlicing is ready 09:32:49,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testLargeReadReplySlicing updated: matchIndex: -1, nextIndex: 0 09:32:49,315 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:49,319 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b2f3c6c: creating a new transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 09:32:49,319 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 09:32:49,319 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:49,319 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:49,319 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] 09:32:49,320 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=0} shard 1 09:32:49,320 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0 09:32:49,320 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:49,320 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:32:49,320 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@7016dc5d 09:32:49,320 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:49,320 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@19beccf8 09:32:49,320 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:49,320 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=12] 09:32:49,320 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing, primaryVersion=12] 09:32:49,320 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@78535e71, attempting to connect 09:32:49,321 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} is not yet known 09:32:49,321 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, nanosAgo=8230, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 09:32:49,322 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:49,322 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:49,322 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:49,322 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@532ccaac instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@77560b5 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@77560b5 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 for transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2299/0x000000080170d058@5626cfcc 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1}, enqueuedTicks=775613678472} 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1}} 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Message size: 449 bytes, total slices to send: 5 09:32:49,323 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 449, offset: 0, size: 100, index: 1 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@22ba6bbe 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=775613678472} 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e cannot transmit request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} - delaying it 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@4b2f3c6c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@79a58e7f 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7aa77071 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@64e94120, splicing queue 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@7aa77071 queue spliced 0 messages 09:32:49,330 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] ClientActorBehavior - member-2-frontend-datastore-testLargeReadReplySlicing: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 7.488 ms 09:32:49,338 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=1, totalSlices=5, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,340 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] - creating AssembledMessageState 09:32:49,341 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 09:32:49,341 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] - expecting more 09:32:49,347 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,347 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 449, offset: 100, size: 100, index: 2 09:32:49,347 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:49,348 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=2, totalSlices=5, lastSliceHashCode=-1085826726, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,348 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1085826726, lastSliceHashCode: -1085826726 09:32:49,348 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] - expecting more 09:32:49,349 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,349 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 449, offset: 200, size: 100, index: 3 09:32:49,349 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=3, totalSlices=5, lastSliceHashCode=-840689321, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: -840689321, lastSliceHashCode: -840689321 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] - expecting more 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 449, offset: 300, size: 100, index: 4 09:32:49,350 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-216411840, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:49,351 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=100, sliceIndex=4, totalSlices=5, lastSliceHashCode=-216411840, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,351 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -216411840, lastSliceHashCode: -216411840 09:32:49,351 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] - expecting more 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 449, offset: 400, size: 49, index: 5 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=-988902345, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], data.length=49, sliceIndex=5, totalSlices=5, lastSliceHashCode=-988902345, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AssembledMessageState - member-1-shard-cars-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex: 5, lastSliceIndex: 4, assembledSize: 400, sliceHashCode: -988902345, lastSliceHashCode: -988902345 09:32:49,352 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] was removed from the cache due to EXPLICIT 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-shard-cars-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446], modifications=1}} 09:32:49,353 AM [cluster-test-shard-dispatcher-12] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testLargeReadReplySlicing: allocating new open transaction member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77], sliceIndex=5, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] 09:32:49,353 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=77] was removed from the cache due to EXPLICIT 09:32:49,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 09:32:49,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=775613678472} 09:32:49,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e transmitted 1 requests 09:32:49,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1}, enqueuedTicks=775613678472, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0}} 09:32:49,354 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=1} completed with ModifyTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=0} 09:32:49,355 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 09:32:49,355 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Message size: 390 bytes, total slices to send: 4 09:32:49,355 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: getNextSlice: total size: 390, offset: 0, size: 100, index: 1 09:32:49,355 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,356 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=1, totalSlices=4, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,356 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received first slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] - creating AssembledMessageState 09:32:49,357 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex: 1, lastSliceIndex: 0, assembledSize: 0, sliceHashCode: -1, lastSliceHashCode: -1 09:32:49,357 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] - expecting more 09:32:49,359 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-886: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=1, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: getNextSlice: total size: 390, offset: 100, size: 100, index: 2 09:32:49,362 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,363 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=2, totalSlices=4, lastSliceHashCode=-1569038920, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,363 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex: 2, lastSliceIndex: 1, assembledSize: 100, sliceHashCode: -1569038920, lastSliceHashCode: -1569038920 09:32:49,363 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] - expecting more 09:32:49,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-886: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=2, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: getNextSlice: total size: 390, offset: 200, size: 100, index: 3 09:32:49,363 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,364 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=100, sliceIndex=3, totalSlices=4, lastSliceHashCode=1439492931, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,364 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex: 3, lastSliceIndex: 2, assembledSize: 200, sliceHashCode: 1439492931, lastSliceHashCode: 1439492931 09:32:49,364 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Added slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] - expecting more 09:32:49,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-886: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=3, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] SlicedMessageState - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: getNextSlice: total size: 390, offset: 300, size: 90, index: 4 09:32:49,366 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Sending next slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-1885386063, replyTo=Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,366 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: handleMessage: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], data.length=90, sliceIndex=4, totalSlices=4, lastSliceHashCode=-1885386063, replyTo=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491]] 09:32:49,366 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AssembledMessageState - member-2-frontend-datastore-testLargeReadReplySlicing: addSlice: identifier: MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex: 4, lastSliceIndex: 3, assembledSize: 300, sliceHashCode: -1885386063, lastSliceHashCode: -1885386063 09:32:49,366 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Received last slice for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: AssembledMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] was removed from the cache due to EXPLICIT 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Notifying callback of re-assembled message Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1475737446], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=775613678472, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1}} 09:32:49,367 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=1} 09:32:49,367 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0} 09:32:49,368 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-2-shard-cars-testLargeReadReplySlicing 09:32:49,368 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 09:32:49,368 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: onShutDown 09:32:49,369 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testLargeReadReplySlicing 09:32:49,369 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testLargeReadReplySlicing (Follower): Closing 09:32:49,369 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 09:32:49,369 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-886: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,369 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: handleMessage: MessageSliceReply [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78], sliceIndex=4, failure=null, sendTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#1475737446]] 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Received last slice reply for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: Removing state for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing_slicer-id-883: SlicedMessageState for MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=883, messageId=78] was removed from the cache due to EXPLICIT 09:32:49,369 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 09:32:49,369 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testLargeReadReplySlicing, generation=0} 09:32:49,369 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testLargeReadReplySlicing: shutting down 09:32:49,369 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testLargeReadReplySlicing: Closing 09:32:49,369 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1475737446] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1475737446] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,369 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: Issuing gracefulStop to shard member-1-shard-cars-testLargeReadReplySlicing 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Shutting down ShardManager shard-manager-testLargeReadReplySlicing - waiting on 1 shards 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: onShutDown 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Initiating leader transfer 09:32:49,370 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,370 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing#1696883142] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@771c6a61 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testLargeReadReplySlicing: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@771c6a61 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Attempting to transfer leadership 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testLargeReadReplySlicing: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testLargeReadReplySlicing: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testLargeReadReplySlicing, leaderId=null, leaderPayloadVersion=12] 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:49,370 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:49,371 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,371 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,431 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491] to Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testLargeReadReplySlicing/member-2-shard-cars-testLargeReadReplySlicing] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:49,669 AM [cluster-test-shard-dispatcher-30] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testLargeReadReplySlicing: Failed to transfer leadership in 299.6 ms 09:32:49,669 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: In unpauseLeader 09:32:49,670 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: leader transfer failed - sending PoisonPill 09:32:49,670 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing (Leader): Leadership transfer expired 09:32:49,670 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - Stopping Shard member-1-shard-cars-testLargeReadReplySlicing 09:32:49,670 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testLargeReadReplySlicing (Leader)_slicer-id-886: Closing 09:32:49,670 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testLargeReadReplySlicing: Aborting 0 pending queued transactions 09:32:49,670 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testLargeReadReplySlicing: All shards shutdown - sending PoisonPill to self 09:32:49,670 AM [cluster-test-shard-dispatcher-19] [INFO] ShardManager - Stopping ShardManager shard-manager-testLargeReadReplySlicing 09:32:49,671 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 09:32:49,671 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testLargeReadReplySlicing: shutting down 09:32:49,671 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testLargeReadReplySlicing: Closing 09:32:49,672 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:49,672 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:32:49.675] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1124)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:49.676] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1124)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:49.676] [cluster-test-akka.remote.default-remote-dispatcher-6] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1124)] [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted [WARN] [04/25/2024 09:32:49.676] [cluster-test-akka.remote.default-remote-dispatcher-7] [akka.stream.Log(akka://cluster-test/system/Materializers/StreamSupervisor-1124)] [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:49,691 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:49,691 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:50,692 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2976832276966484022] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:50,711 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:50,711 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:50,713 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [3604784638137504102] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated [WARN] [04/25/2024 09:32:50.713] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:50,720 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:50,720 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:32:50.723] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:50,740 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:50,746 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [8844680016264701131] 09:32:50,746 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:50,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:50,747 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:50,758 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:50,758 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:50,758 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:50,758 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:50,759 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:50,760 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:50,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-354906381607186721] 09:32:50,764 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:50,765 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:50,765 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:50,766 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:50,766 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:50,781 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:50,798 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-4110660762494924547] 09:32:50,798 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:50,803 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:50,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:50,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:50,803 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:50,812 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:50,813 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:50,814 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 09:32:50,848 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1979562567] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,848 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:50,851 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:50,852 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:50,856 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:50,856 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,857 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:50,857 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@43a338c to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@abe8c93 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@e81091c 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1714037570876, None)) 09:32:50,876 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037570875,0,0)) 09:32:50,877 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@abe8c93 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4a6e7b6 09:32:50,877 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6db90661 09:32:50,877 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 09:32:50,877 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:50,879 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:50,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:50,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,880 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,880 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] 09:32:50,881 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 09:32:50,881 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,881 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#100460306] 09:32:50,881 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-1:snapshot-read#100460306] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,888 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#531955620 created and ready for shard:member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,888 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 09:32:50,889 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleTransactionsWritesInQuickSuccession 09:32:50,890 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@532ccaac aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@371d0ec6 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1} 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@3bfdfeba 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testLargeReadReplySlicing: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testLargeReadReplySlicing, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testLargeReadReplySlicing/member-1-shard-cars-testLargeReadReplySlicing#2020084491], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:50,890 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@14a8905e transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=0, protocol=ABORT}, enqueuedTicks=777183594274} 09:32:50,890 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: slice: identifier: member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1475737446], modifications=0, protocol=ABORT}} 09:32:50,890 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Message size: 275 bytes, total slices to send: 3 09:32:50,890 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: getNextSlice: total size: 275, offset: 0, size: 100, index: 1 09:32:50,890 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testLargeReadReplySlicing_slicer-id-884: Sending first slice: MessageSlice [identifier=MessageSliceIdentifier [clientIdentifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-1, slicerId=884, messageId=79], data.length=100, sliceIndex=1, totalSlices=3, lastSliceHashCode=-1, replyTo=Actor[akka://cluster-test/user/$a#1475737446]] 09:32:50,890 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientTransaction{identifier=member-2-datastore-testLargeReadReplySlicing-fe-0-txn-0-0} 09:32:50,890 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@26b56c1a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 1 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@450174c2 09:32:50,890 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@521ed5e4 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT}, enqueuedTicks=777184054991} 09:32:50,890 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT}} 09:32:50,890 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message size: 293 bytes, total slices to send: 1 09:32:50,890 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message does not need to be sliced - sending original message 09:32:50,890 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@6cebee5e aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 1 09:32:50,890 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@18ee61f0 09:32:50,890 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@4e211a93 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT}, enqueuedTicks=777184239466} 09:32:50,890 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: slice: identifier: member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1076505859], modifications=0, protocol=ABORT}} 09:32:50,891 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message size: 294 bytes, total slices to send: 1 09:32:50,891 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testReadWriteTransactionWithMultipleShards_slicer-id-877: Message does not need to be sliced - sending original message 09:32:50,891 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testReadWriteTransactionWithMultipleShards-fe-0-txn-1-0} 09:32:50,891 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@47de929a aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@223b69ce 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1} 09:32:50,891 AM [Cleaner-0] [DEBUG] RemoteProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} has a successor, which should receive seal through a separate request 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1920087265], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@45b2f159 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-1-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#1018090343], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:50,891 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1d3d4ef3 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1920087265]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@199214d2 09:32:50,891 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionRetryWithInitialAskTimeoutExOnCreateTx/member-2-shard-cars-testTransactionRetryWithInitialAskTimeoutExOnCreateTx#922677164], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:50,891 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionRetryWithInitialAskTimeoutExOnCreateTx-fe-0-txn-0-0} 09:32:50,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:50,891 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:50,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 09:32:50,891 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 09:32:50,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:32:50,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:32:50,892 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 09:32:50,892 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 09:32:50,892 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 09:32:50,894 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#531955620] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,911 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,911 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:50,921 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: no initial generation override, starting from 0 09:32:50,921 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:32:50,921 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@1b93f8aa to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76ec00b1 09:32:50,921 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@72d38ab4 09:32:50,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:50,921 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Creating local shard: member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:50,921 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,921 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, 0, 1714037570921, None)) 09:32:50,922 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037570920,0,0)) 09:32:50,922 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@76ec00b1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1053e875 09:32:50,922 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} started 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@64317d3 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: updated peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession on shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#53978448] 09:32:50,923 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, persistent : true 09:32:50,924 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with journal batch size 1 09:32:50,924 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession#53978448] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,924 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1351430660 created and ready for shard:member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,924 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#518467406] 09:32:50,924 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-notifier#1351430660] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,928 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/shard-member-2:snapshot-read#518467406] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:50,932 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Initializing shard [cars] 09:32:50,932 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession is initialized 09:32:50,932 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from null to Follower 09:32:50,932 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession , registered listener akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession 09:32:50,935 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:50,937 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession from null to Follower 09:32:50,973 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1979562567] 09:32:50,973 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 09:32:50,973 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Received ElectionTimeout - switching to Candidate 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 1 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: false, hasLeader : false 09:32:51,117 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession is no longer the leader 09:32:51,117 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Follower to Candidate 09:32:51,118 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Follower to Candidate 09:32:51,376 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:32:51,376 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,376 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 2 09:32:51,498 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:51,667 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:32:51,667 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,667 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 3 09:32:51,768 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:51,768 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,768 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,768 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-3-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,773 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession to akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,819 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:51,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Received ElectionTimeout 09:32:51,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,967 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Starting new election term 4 09:32:51,967 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,968 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:51,968 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Cluster state: [] 09:32:51,968 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:51,968 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:51,968 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=null 09:32:51,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:51,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,969 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onLeadershipChange, isLeader: true, hasLeader : true 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: became leader with frontend state for [] 09:32:51,970 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession , received role change from Candidate to Leader 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=12] 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received role changed for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession from Candidate to Leader 09:32:51,970 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Closing 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: getLeaderAddress leaderId = member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done false 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderPayloadVersion=12] 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:51,971 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All Shards are ready - data store testSingleTransactionsWritesInQuickSuccession is ready 09:32:51,971 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: -1, nextIndex: 0 09:32:51,971 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:51,977 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b 09:32:51,977 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:51,977 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 09:32:51,977 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:51,977 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:51,977 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] 09:32:51,977 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:51,978 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=12] 09:32:51,978 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} shard 1 09:32:51,978 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, primaryVersion=12] 09:32:51,978 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:51,978 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:32:51,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:51,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@484745c3 09:32:51,978 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:51,978 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-0 09:32:51,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 09:32:51,978 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@51ff818 09:32:51,979 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6ed0e29c, attempting to connect 09:32:51,980 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} is not yet known 09:32:51,980 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, nanosAgo=7700, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:32:51,981 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23c7756b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778271408646} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@6349056b 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY}, enqueuedTicks=778271716171} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY}} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 416 bytes, total slices to send: 1 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2302/0x000000080170df70@769cef43 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778271839543} 09:32:51,981 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 296 bytes, total slices to send: 1 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@a66009d 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@15994b65 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2328f6e2, splicing queue 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@15994b65 queue spliced 0 messages 09:32:51,982 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.030 ms 09:32:51,982 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 09:32:51,982 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 09:32:51,982 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:32:51,982 AM [cluster-test-shard-dispatcher-22] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@25109afa 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 ready 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 to ready 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 is already in state READY (READY) 09:32:51,983 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct canCommit 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 validated 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct preCommit 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 candidate ready 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: canCommit - messages: [] 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 prepared 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 initiating direct commit 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 184 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 submitted to persistence 09:32:51,984 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 09:32:51,985 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 09:32:51,985 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:51,985 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY}, enqueuedTicks=778271716171, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:51,985 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=0} 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}]]] 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=184} 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:32:51,985 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:51,986 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75}] 09:32:51,986 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=184}] 09:32:51,986 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession Received follower initial sync status for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession status sync done true 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 1, nextIndex: 2 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 0 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2178/0x000000080169b4b8@11aa7875 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] LeaderFrontendState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7fa61017}, purged=MutableUnsignedLongSet{size=0}} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 1 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:32:51,986 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1116960910] with name member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 09:32:51,987 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 09:32:51,987 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:51,987 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1116960910] 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#1116960910] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 59.55 μs 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 09:32:51,987 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778271408646, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0}} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, sequence=0} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 1 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778271839543, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 0 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from -1 to 0 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=1} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1}, size=75} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@37c3ea24}, purged=MutableUnsignedLongSet{size=0}} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 1 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 0 to 1 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 1 data Simple{size=184} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4736a6fc 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7d6e346f 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778281437727} 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770171580] with name member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770171580] 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 0 for term 4 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:51,988 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=1 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 65.63 μs 09:32:51,988 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher#770171580] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:51,989 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:51,989 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:32:51,989 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 09:32:51,990 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@12138ad 09:32:51,990 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778283347119} 09:32:51,990 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}} 09:32:51,990 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 456 bytes, total slices to send: 1 09:32:51,990 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,990 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:51,990 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-0 09:32:51,990 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} allocated sequence 1 09:32:51,990 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6111a60d 09:32:51,990 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778283639154} 09:32:51,990 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=5, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:51,990 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:51,990 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@2354ea26 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 ready 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE}}}}}}}]} 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 to ready 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 is already in state READY (READY) 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct canCommit 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 validated 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:51,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct preCommit 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 candidate ready 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - candidate rootPath: / 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: canCommit - messages: [] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: preCommit - successfulFromPrevious: [] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 prepared 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 initiating direct commit 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 submitted to persistence 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 09:32:51,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 09:32:51,992 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 09:32:51,992 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778283347119, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0}} 09:32:51,992 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=0} 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}]]] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76}] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=241}] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 3, nextIndex: 4 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 2 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@531cfc6c 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 3 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1: commit - successfulFromPrevious: [] 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:32:51,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=APPEARED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car0}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 41.40 μs 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778281437727, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=DONE} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: purge completed 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@23c7756b purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778283639154, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=1}} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED} directCommit completed 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=FLUSHED}: initiating purge 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} allocated sequence 2 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@53b04e92 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:51,994 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778288055900} 09:32:51,994 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-0 09:32:51,994 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 09:32:51,994 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@7424e5c8 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 3 09:32:51,994 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778288185432} 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 2 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 1 to 2 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1, size=76} 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-0-1 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 3 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 2 to 3 09:32:51,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 3 data Simple{size=241} 09:32:51,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 2 for term 4 09:32:51,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=3 09:32:51,995 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:51,995 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 31.48 μs 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}]]] 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@bc22ae8 09:32:51,996 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:51,996 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 388 bytes, total slices to send: 1 09:32:51,996 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:51,996 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}]]] 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 3, entries to be added from: 0 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:32:51,997 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=4, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:51,998 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:51,998 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778288185432, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:51,998 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=0} 09:32:51,998 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 09:32:51,998 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77}] 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@7d3c05e5 09:32:51,998 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778291526849} 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:51,998 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,998 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:51,998 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} allocated sequence 0 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@4a420163 09:32:51,998 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778291886145} 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=9, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}} 09:32:51,998 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 456 bytes, total slices to send: 1 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:51,998 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:51,998 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-0 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} allocated sequence 1 09:32:51,998 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@101fcf9e 09:32:51,998 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778292138879} 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:51,998 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:51,998 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 4, nextIndex: 5 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 4, lastAppliedIndex: 3 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 4 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@23d03fc0 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 4, leaderSnapShotIndex: 3 09:32:52,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 09:32:52,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@bc22ae8 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 ready 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE}}}}}}}]} 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 to ready 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 is already in state READY (READY) 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct canCommit 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 validated 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct preCommit 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 candidate ready 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - candidate rootPath: / 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: canCommit - messages: [] 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: preCommit - successfulFromPrevious: [] 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 prepared 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 initiating direct commit 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 5, leaderLastIndex: 6, leaderSnapShotIndex: 3 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 submitted to persistence 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778288055900, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, sequence=2}} 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE}: purge completed 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, state=DONE} 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 4 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 4, context.getLastApplied(): 3, lastIndex(): 4 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 4 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 3 to 4 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 4 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1, size=77} 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-1-1 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 3 for term 4 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=4, prevLogTerm=4, leaderCommit=4, replicatedToAllIndex=3, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}]]] 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 4, entries to be added from: 0 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=6, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778291886145, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0}} 09:32:52,4 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=0} 09:32:52,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 09:32:52,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=Simple{size=241}] 09:32:52,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=4 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 5 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 6, nextIndex: 7 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 6, lastAppliedIndex: 4 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 5 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2307/0x000000080171d3e0@64365b32 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 6 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1: commit - successfulFromPrevious: [] 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 changing state from COMMIT_PENDING to COMMITTED 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car1}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 6, leaderSnapShotIndex: 5 09:32:52,6 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 09:32:52,6 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,6 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 37.24 μs 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778291526849, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1}} 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=1} 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=OPEN}: initiating purge 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 2 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7f07a693 09:32:52,7 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778301227643} 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 6 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 6, context.getLastApplied(): 4, lastIndex(): 6 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 5 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 4 to 5 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 5 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 6 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 5 to 6 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 6 data Simple{size=241} 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 5 for term 4 09:32:52,8 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=6 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 30.03 μs 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778292138879, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=1}} 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED} directCommit completed 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=FLUSHED}: initiating purge 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} allocated sequence 2 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@324c03f7 09:32:52,8 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778302204700} 09:32:52,9 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-0 09:32:52,9 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 0 09:32:52,9 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@61001ed7 09:32:52,9 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778302390954} 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, logIndex: 7, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, logIndex: 8, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@c30435a 09:32:52,10 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 09:32:52,10 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,10 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 408 bytes, total slices to send: 1 09:32:52,10 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:52,11 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 09:32:52,11 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 09:32:52,11 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778302390954, sessionId=0, txSequence=13} with Envelope{sessionId=0, txSequence=d, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0}} 09:32:52,11 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=0} 09:32:52,11 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} allocated sequence 1 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@49493e99 09:32:52,12 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778305348404} 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, message: Envelope{sessionId=0, txSequence=e, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:52,12 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,12 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:52,12 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} allocated sequence 0 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@61a0e649 09:32:52,12 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778305682369} 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}} 09:32:52,12 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 456 bytes, total slices to send: 1 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,12 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:52,12 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-0 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} allocated sequence 1 09:32:52,12 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@53eeb152 09:32:52,12 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778305885133} 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, message: Envelope{sessionId=0, txSequence=10, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:52,12 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,12 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 9, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,13 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@c30435a 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 ready 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE}}}}}}}]} 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 to ready 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 is already in state READY (READY) 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct canCommit 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 validated 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct preCommit 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 candidate ready 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - candidate rootPath: / 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: canCommit - messages: [] 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: preCommit - successfulFromPrevious: [] 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 prepared 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 initiating direct commit 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 10, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 7, leaderLastIndex: 10, leaderSnapShotIndex: 5 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 7 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 submitted to persistence 09:32:52,14 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,14 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 09:32:52,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 09:32:52,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778305682369, sessionId=0, txSequence=15} with Envelope{sessionId=0, txSequence=f, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0}} 09:32:52,15 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=0} 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=6, prevLogTerm=4, leaderCommit=6, replicatedToAllIndex=5, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}], SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}], SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}]]] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 6, entries to be added from: 0 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=10, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=7, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=8, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=9, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,15 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=10, term=4, payload=Simple{size=241}] 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 10, nextIndex: 11 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 10, lastAppliedIndex: 6 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 7 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@64f69844 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 8 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@4ec205c8 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 9 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2307/0x000000080171d3e0@665d15fb 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 10 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1: commit - successfulFromPrevious: [] 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 changing state from COMMIT_PENDING to COMMITTED 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car2}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:32:52,16 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 09:32:52,16 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 53.70 μs 09:32:52,16 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 10, leaderSnapShotIndex: 9 09:32:52,17 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778301227643, sessionId=0, txSequence=11} with Envelope{sessionId=0, txSequence=b, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, sequence=2}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE}: purge completed 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, state=DONE} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778302204700, sessionId=0, txSequence=12} with Envelope{sessionId=0, txSequence=c, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, sequence=2}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE}: purge completed 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, state=DONE} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778305348404, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=1} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=OPEN}: initiating purge 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} allocated sequence 2 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@67c9a2c 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778310935499} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778305885133, sessionId=0, txSequence=16} with Envelope{sessionId=0, txSequence=10, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=1}} 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED} directCommit completed 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=FLUSHED}: initiating purge 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} allocated sequence 2 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@1757b994 09:32:52,17 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778311096031} 09:32:52,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 10 09:32:52,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 10, context.getLastApplied(): 6, lastIndex(): 10 09:32:52,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 7 09:32:52,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 6 to 7 09:32:52,17 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 7 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1, size=77} 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-2-1 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 8 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 7 to 8 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 8 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1, size=77} 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-3-1 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 9 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 8 to 9 09:32:52,18 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-0 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 9 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,18 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 0 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 10 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 9 to 10 09:32:52,18 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@6a9a677 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 10 data Simple{size=241} 09:32:52,18 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778311515308} 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 9 for term 4 09:32:52,18 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=10 09:32:52,19 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,19 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 33.27 μs 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, logIndex: 11, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, logIndex: 12, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,19 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@100743e5 09:32:52,20 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 09:32:52,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,20 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 09:32:52,20 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 431 bytes, total slices to send: 1 09:32:52,20 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:52,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 09:32:52,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778311515308, sessionId=0, txSequence=19} with Envelope{sessionId=0, txSequence=13, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0}} 09:32:52,21 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=0} 09:32:52,21 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} allocated sequence 1 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@23f04368 09:32:52,21 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778314455078} 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, message: Envelope{sessionId=0, txSequence=14, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:52,21 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,21 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:52,21 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} allocated sequence 0 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@34c68277 09:32:52,21 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778314755243} 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=15, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}} 09:32:52,21 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 456 bytes, total slices to send: 1 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,21 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:52,21 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-0 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} allocated sequence 1 09:32:52,21 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@542a6430 09:32:52,21 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778314951647} 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, message: Envelope{sessionId=0, txSequence=16, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:52,21 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,21 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 13, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@100743e5 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 ready 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE}}}}}}}]} 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 to ready 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 is already in state READY (READY) 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct canCommit 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 validated 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct preCommit 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 candidate ready 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - candidate rootPath: / 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: canCommit - messages: [] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: preCommit - successfulFromPrevious: [] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 prepared 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 initiating direct commit 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 14, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 11, leaderLastIndex: 14, leaderSnapShotIndex: 9 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 11 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 submitted to persistence 09:32:52,23 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,24 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 09:32:52,24 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 09:32:52,24 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778314755243, sessionId=0, txSequence=21} with Envelope{sessionId=0, txSequence=15, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0}} 09:32:52,24 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=0} 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=10, prevLogTerm=4, leaderCommit=10, replicatedToAllIndex=9, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}], SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}], SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}]]] 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 10, entries to be added from: 0 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 09:32:52,24 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=14, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=11, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77}] 09:32:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=12, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77}] 09:32:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=13, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,25 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=14, term=4, payload=Simple{size=241}] 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 14, nextIndex: 15 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 14, lastAppliedIndex: 10 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 11 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@11b4a91 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 12 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@54375045 09:32:52,25 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 13 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2307/0x000000080171d3e0@6e8ade8e 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 14 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1: commit - successfulFromPrevious: [] 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 changing state from COMMIT_PENDING to COMMITTED 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car3}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:32:52,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 09:32:52,26 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 75.18 μs 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 14, leaderSnapShotIndex: 13 09:32:52,26 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 14 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 14, context.getLastApplied(): 10, lastIndex(): 14 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 11 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 10 to 11 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 11 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, size=77} 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 12 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 11 to 12 09:32:52,28 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 12 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, size=77} 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 13 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 12 to 13 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 13 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 14 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 13 to 14 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 14 data Simple{size=241} 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 13 for term 4 09:32:52,29 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=14 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778310935499, sessionId=0, txSequence=17} with Envelope{sessionId=0, txSequence=11, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, sequence=2}} 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE}: purge completed 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-4-1, state=DONE} 09:32:52,29 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778311096031, sessionId=0, txSequence=18} with Envelope{sessionId=0, txSequence=12, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, sequence=2}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE}: purge completed 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-5-1, state=DONE} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 127.7 μs 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778314455078, sessionId=0, txSequence=20} with Envelope{sessionId=0, txSequence=14, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=1} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=OPEN}: initiating purge 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} allocated sequence 2 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@1ce2d3d7 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778323456131} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778314951647, sessionId=0, txSequence=22} with Envelope{sessionId=0, txSequence=16, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=1}} 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED} directCommit completed 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=FLUSHED}: initiating purge 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} allocated sequence 2 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@26f2a0bd 09:32:52,30 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778323665105} 09:32:52,30 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-0 09:32:52,30 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 0 09:32:52,30 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@685fd9f0 09:32:52,30 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778324097822} 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, logIndex: 15, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, logIndex: 16, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,31 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@35179a61 09:32:52,31 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 09:32:52,32 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,32 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 451 bytes, total slices to send: 1 09:32:52,32 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:52,32 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 09:32:52,33 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 09:32:52,33 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778324097822, sessionId=0, txSequence=25} with Envelope{sessionId=0, txSequence=19, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0}} 09:32:52,33 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=0} 09:32:52,33 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} 09:32:52,33 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} allocated sequence 1 09:32:52,33 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@4d25d31d 09:32:52,33 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778326747777} 09:32:52,33 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, message: Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:52,33 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,33 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,33 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from IDLE to TX_OPEN 09:32:52,33 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:32:52,33 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} allocated sequence 0 09:32:52,33 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@1db6f6bb 09:32:52,33 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778327237806} 09:32:52,33 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}} 09:32:52,34 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 456 bytes, total slices to send: 1 09:32:52,34 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,34 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b changed state from TX_OPEN to IDLE 09:32:52,34 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@490a590b readied transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-0 09:32:52,34 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} allocated sequence 1 09:32:52,34 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6589e0fb 09:32:52,34 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778327558562} 09:32:52,34 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, message: Envelope{sessionId=0, txSequence=1c, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}} 09:32:52,34 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,34 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 17, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@35179a61 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: readying transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 ready 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE}}}}}}}]} 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: transitioned member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 to ready 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 is already in state READY (READY) 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct canCommit 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Validating transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 validated 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct preCommit 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Preparing transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 candidate ready 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - candidate rootPath: / 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: canCommit - messages: [] 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: preCommit - successfulFromPrevious: [] 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 prepared 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 initiating direct commit 09:32:52,35 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Starting commit for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 18, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 15, leaderLastIndex: 18, leaderSnapShotIndex: 13 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 15 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 09:32:52,36 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 submitted to persistence 09:32:52,36 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 09:32:52,36 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY}, enqueuedTicks=778327237806, sessionId=0, txSequence=27} with Envelope{sessionId=0, txSequence=1b, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0}} 09:32:52,36 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=0} 09:32:52,37 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=14, prevLogTerm=4, leaderCommit=14, replicatedToAllIndex=13, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}], SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}], SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}]]] 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 4 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 14, entries to be added from: 0 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,37 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log Simple{size=241} 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 5 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=18, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=15, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=16, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=17, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=18, term=4, payload=Simple{size=241}] 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 18, nextIndex: 19 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 18, lastAppliedIndex: 14 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 15 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@6cf3cfbc 09:32:52,38 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 16 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@60865b 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 17 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2307/0x000000080171d3e0@1ac310e7 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 18 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Resuming commit of transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1: commit - successfulFromPrevious: [] 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 changing state from COMMIT_PENDING to COMMITTED 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=car4}], operation=WRITE, modificationType=WRITE}}}}}}}]} 09:32:52,39 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 09:32:52,39 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 71.30 μs 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 19, leaderLastIndex: 18, leaderSnapShotIndex: 17 09:32:52,39 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778323456131, sessionId=0, txSequence=23} with Envelope{sessionId=0, txSequence=17, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, sequence=2}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE}: purge completed 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, state=DONE} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778323665105, sessionId=0, txSequence=24} with Envelope{sessionId=0, txSequence=18, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, sequence=2}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE}: purge completed 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, state=DONE} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778326747777, sessionId=0, txSequence=26} with Envelope{sessionId=0, txSequence=1a, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=1} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=OPEN}: initiating purge 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} allocated sequence 2 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@628d49d1 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778333589734} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=SIMPLE}, enqueuedTicks=778327558562, sessionId=0, txSequence=28} with Envelope{sessionId=0, txSequence=1c, message=TransactionCommitSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=1}} 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED} directCommit completed 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=FLUSHED}: initiating purge 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} allocated sequence 2 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@33985fcc 09:32:52,40 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778333769117} 09:32:52,40 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-0 09:32:52,40 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 0 09:32:52,40 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@3e118047 09:32:52,40 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778333950880} 09:32:52,40 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 18 09:32:52,40 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 18, context.getLastApplied(): 14, lastIndex(): 18 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 15 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 14 to 15 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 15 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1, size=77} 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-6-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 16 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 15 to 16 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 16 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1, size=77} 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-7-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 17 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 16 to 17 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 17 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 18 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 17 to 18 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 18 data Simple{size=241} 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying foreign transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, logIndex: 19, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Committed transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 17 for term 4 09:32:52,41 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=18 09:32:52,41 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: publishChanges: rootPath: / 09:32:52,41 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Elapsed time for generation of change events for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession-DTCL-publisher: 36.04 μs 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, logIndex: 20, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 19 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 09:32:52,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@77f51a10 09:32:52,43 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 09:32:52,43 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 09:32:52,43 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 471 bytes, total slices to send: 1 09:32:52,43 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:52,44 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 09:32:52,44 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=778333950880, sessionId=0, txSequence=31} with Envelope{sessionId=0, txSequence=1f, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0}} 09:32:52,44 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=0} 09:32:52,44 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} 09:32:52,44 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} allocated sequence 1 09:32:52,44 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@762463b3 09:32:52,44 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778337741084} 09:32:52,44 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, message: Envelope{sessionId=0, txSequence=20, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:52,44 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,44 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=18, prevLogTerm=4, leaderCommit=18, replicatedToAllIndex=17, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}], SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}]]] 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 2 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 18, entries to be added from: 0 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 3 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=20, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 21, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,45 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=19, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77}] 09:32:52,45 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=20, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77}] 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 20, nextIndex: 21 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 20, lastAppliedIndex: 18 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 19 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@271d967b 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 20 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@39600999 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 21, leaderLastIndex: 21, leaderSnapShotIndex: 19 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 21 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 09:32:52,46 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778333589734, sessionId=0, txSequence=29} with Envelope{sessionId=0, txSequence=1d, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, sequence=2}} 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE}: purge completed 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, state=DONE} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=20, prevLogTerm=4, leaderCommit=20, replicatedToAllIndex=19, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 20, entries to be added from: 0 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778333769117, sessionId=0, txSequence=30} with Envelope{sessionId=0, txSequence=1e, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, sequence=2}} 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE}: purge completed 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 4 09:32:52,47 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, state=DONE} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 20 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=21, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 20, context.getLastApplied(): 18, lastIndex(): 21 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 19 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 18 to 19 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 19 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1, size=77} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-8-1 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 20 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 19 to 20 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 20 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1, size=77} 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-9-1 09:32:52,47 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 19 for term 4 09:32:52,48 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=21, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,48 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=20 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 21, nextIndex: 22 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 21, lastAppliedIndex: 20 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 21 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.FrontendReadOnlyTransaction$$Lambda$2307/0x000000080171d3e0@3af9a4b4 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 22, leaderLastIndex: 21, leaderSnapShotIndex: 20 09:32:52,48 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 21 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778337741084, sessionId=0, txSequence=32} with Envelope{sessionId=0, txSequence=20, message=ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1}} 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 21, context.getLastApplied(): 20, lastIndex(): 21 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 abort completed with ModifyTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=1} 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 21 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 20 to 21 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=OPEN}: initiating purge 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 21 data AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} allocated sequence 2 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborted transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 20 for term 4 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@51571e69 09:32:52,49 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=21 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:52,49 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778342559106} 09:32:52,49 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,50 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,50 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,50 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, logIndex: 22, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:52,50 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 22 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,206 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=21, prevLogTerm=4, leaderCommit=21, replicatedToAllIndex=20, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}]]] 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Number of entries to be appended = 1 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): After cleanup, lastIndex: 21, entries to be added from: 0 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Log size is now 2 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession, logLastIndex=22, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:52,208 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=22, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77}] 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession updated: matchIndex: 22, nextIndex: 23 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Applying to log - commitIndex: 22, lastAppliedIndex: 21 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Setting last applied to 22 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,208 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: replication of member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@2f841833 09:32:52,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: finished purging transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 09:32:52,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 22, leaderSnapShotIndex: 21 09:32:52,209 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Commit index set to 22 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 22, context.getLastApplied(): 21, lastIndex(): 22 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Setting last applied to 22 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Moving last applied index from 21 to 22 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Applying state for log index 22 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, size=77} 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Purged transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: fakeSnapshot purging log to 21 for term 4 09:32:52,209 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persisting ApplyJournalEntries with index=22 09:32:52,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 09:32:52,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#1630193646]}, enqueuedTicks=778342559106, sessionId=0, txSequence=33} with Envelope{sessionId=0, txSequence=21, message=TransactionPurgeResponse{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, sequence=2}} 09:32:52,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE}: purge completed 09:32:52,209 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-10-1, state=DONE} 09:32:52,545 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#1979562567] 09:32:52,545 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: findLocalShard : cars 09:32:52,545 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] 09:32:52,545 AM [awaitility[Range set leak test]] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.TestShard$RequestFrontendMetadata to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] 09:32:52,546 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 for transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-0 09:32:52,546 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 0 09:32:52,546 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@2aae83c9 09:32:52,546 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:52,546 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=778839938836} 09:32:52,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: allocating new open snapshot member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 09:32:52,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,547 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@77f51a10 09:32:52,547 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 09:32:52,547 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SlicedMessageState - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message size: 460 bytes, total slices to send: 1 09:32:52,547 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession_slicer-id-888: Message does not need to be sliced - sending original message 09:32:52,548 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 09:32:52,548 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1630193646], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=778839938836, sessionId=0, txSequence=34} with Envelope{sessionId=0, txSequence=22, message=ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0}} 09:32:52,548 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=0} 09:32:52,549 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@2f692e43 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} 09:32:52,549 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} allocated sequence 1 09:32:52,549 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@7538a3b4 09:32:52,549 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:52,549 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@322ff8b9 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}, enqueuedTicks=778842385477} 09:32:52,549 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: slice: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, message: Envelope{sessionId=0, txSequence=23, message=ModifyTransactionRequest{target=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1630193646], modifications=0, protocol=ABORT}} 09:32:52,549 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message size: 297 bytes, total slices to send: 1 09:32:52,549 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession_slicer-id-889: Message does not need to be sliced - sending original message 09:32:52,549 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:32:52,549 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,549 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 09:32:52,549 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 09:32:52,549 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: aborting transaction member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1 09:32:52,549 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Persist data SimpleReplicatedLogEntry [index=23, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, size=77}] 09:32:52,549 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Append log entry and persist SimpleReplicatedLogEntry [index=23, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, size=77}] 09:32:52,549 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,549 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Replicate message: identifier: member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, logIndex: 23, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:52,549 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession (Follower): Closing 09:32:52,549 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 09:32:52,550 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:52,550 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:32:52,550 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: persist complete SimpleReplicatedLogEntry [index=23, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, size=77}] 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Issuing gracefulStop to shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientHistory - Force-closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=0} java.lang.IllegalStateException: Client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0} has been shut down at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior.shutdown(AbstractDataStoreClientBehavior.java:113) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:148) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession - waiting on 1 shards 09:32:52,550 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: onShutDown 09:32:52,550 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Initiating leader transfer 09:32:52,550 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1630193646] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1630193646] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:52,550 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ebb7d08 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, leaderId=null, leaderPayloadVersion=12] 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:52,550 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleTransactionsWritesInQuickSuccession, generation=0}, history=1, cookie=1} 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@7ebb7d08 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Attempting to transfer leadership 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: active: true, followerNextIndex: 23, leaderLastIndex: 23, leaderSnapShotIndex: 21 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 23 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,551 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession: AppendEntries [leaderId=member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession, prevLogIndex=22, prevLogTerm=4, leaderCommit=22, replicatedToAllIndex=21, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=23, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testSingleTransactionsWritesInQuickSuccession-fe-0-chn-1-txn-11-1, size=77}]]] 09:32:52,551 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession#2134179302] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:52,552 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:52,553 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession#2010893575] to Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleTransactionsWritesInQuickSuccession/member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:52,567 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): sendAppendEntries: 23 is present for follower member-2-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,787 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Failed to transfer leadership in 256.5 ms 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: In unpauseLeader 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: leader transfer failed - sending PoisonPill 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader): Leadership transfer expired 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession 09:32:52,807 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession (Leader)_slicer-id-891: Closing 09:32:52,808 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testSingleTransactionsWritesInQuickSuccession: Aborting 0 pending queued transactions 09:32:52,808 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleTransactionsWritesInQuickSuccession: All shards shutdown - sending PoisonPill to self 09:32:52,808 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleTransactionsWritesInQuickSuccession 09:32:52,808 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:32:52,808 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: shutting down 09:32:52,808 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleTransactionsWritesInQuickSuccession: Closing 09:32:52,809 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:52,809 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:52,813 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:52,813 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:52,813 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:52,813 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:52,815 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:52,829 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:52,829 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:53,828 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [8844680016264701131] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:53,848 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-354906381607186721] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:53,851 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:53,851 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:53,852 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:53.854] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:53,859 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:53,859 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:53,860 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:53.861] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:53,875 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:53,887 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4050052283174432206] 09:32:53,887 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:53,888 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:53,888 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:53,891 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:53,891 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:53,904 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:53,907 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [841458959469681748] 09:32:53,907 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:53,909 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:53,909 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:53,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:53,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:53,909 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:53,914 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:53,915 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:53,922 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:53,929 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [6821463735254649215] 09:32:53,929 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:53,939 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:53,939 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:53,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:53,941 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:53,942 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:53,944 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:53,946 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:53,951 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,951 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@6fa27e59 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74a6f4a6 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@33b0ceb9 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037573959, None)) 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037573958,0,0)) 09:32:53,959 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@74a6f4a6 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5825dbd 09:32:53,959 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:32:53,960 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:53,961 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:53,962 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:53,963 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:53,964 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:53,970 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:53,970 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,971 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:53,971 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,971 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@257bcef4 09:32:53,971 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:53,971 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,975 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,983 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:53,984 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,984 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,984 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] 09:32:53,984 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Shard created : member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:32:53,984 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Starting recovery for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:32:53,985 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,985 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-72377454 created and ready for shard:member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,985 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-72377454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,985 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#391259076] 09:32:53,985 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-1:snapshot-read#391259076] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:53,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@9b6a595 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d98423 09:32:53,986 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7e61080 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:32:53,986 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:32:53,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:53,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:53,986 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037573986, None)) 09:32:53,987 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037573985,0,0)) 09:32:53,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7b6d38a8 09:32:53,987 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3d98423 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@1f98264b 09:32:53,987 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:32:53,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:53,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,991 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:32:53,995 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#310397543] 09:32:53,996 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:32:53,996 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:32:53,996 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop#310397543] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,996 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-651967321] 09:32:53,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-922678896 created and ready for shard:member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-922678896] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,996 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-2:snapshot-read#-651967321] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:53,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:32:53,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:53,997 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:53,997 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:53,997 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:32:54,12 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:54,14 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:54,37 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] 09:32:54,37 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: findLocalShard : cars 09:32:54,38 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] 09:32:54,272 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:54,272 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=1, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 1 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:54,273 AM [cluster-test-akka.actor.default-dispatcher-20] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 09:32:54,273 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 09:32:54,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:32:54,572 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=2, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,572 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 2 09:32:54,606 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:54,822 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:32:54,822 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=3, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,822 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 3 09:32:54,903 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:54,903 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,904 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,908 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,908 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:54,960 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:55,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Received ElectionTimeout 09:32:55,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,52 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 4 09:32:55,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,52 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,53 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.RequestVote] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=null 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=4, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,54 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:55,54 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:55,54 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:55,54 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:32:55,54 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [] 09:32:55,55 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to Leader 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to Leader 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:32:55,55 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,55 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:32:55,55 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 09:32:55,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 09:32:55,61 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:55,70 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:32:55,70 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 09:32:55,70 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,70 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,70 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] 09:32:55,70 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} shard 1 09:32:55,70 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:32:55,70 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:55,70 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-0 09:32:55,71 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@4531f70f 09:32:55,71 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:55,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:55,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@271d7e4e] 09:32:55,71 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@271d7e4e] 09:32:55,71 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@377e2b2f, attempting to connect 09:32:55,71 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} is not yet known 09:32:55,72 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, nanosAgo=12320, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:55,77 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:55,77 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:55,77 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@37ba099a instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fdd6351 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fdd6351 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=OPEN} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], modifications=1, protocol=SIMPLE} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} allocated sequence 0 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@4531f70f 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5bff22ed transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false}, enqueuedTicks=781364329473} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1ffc1528 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@af01781 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@59981c4a, splicing queue 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@af01781 queue spliced 0 messages 09:32:55,78 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 862.9 μs 09:32:55,78 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct canCommit 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 validated 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct preCommit 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 candidate ready 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: canCommit - messages: [] 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 prepared 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 initiating direct commit 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,79 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:55,80 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 submitted to persistence 09:32:55,80 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,82 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}]]] 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 1 09:32:55,82 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:55,85 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,85 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 0, nextIndex: 1 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 0 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1af6a31b}, purged=MutableUnsignedLongSet{size=0}} 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:32:55,86 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:55,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1929317560] with name member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:32:55,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:32:55,87 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false}, enqueuedTicks=781364329473, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=0}} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1929317560] 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7b52a5de 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SEALED}: initiating purge 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} allocated sequence 1 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4f8bac9 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5bff22ed transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]}, enqueuedTicks=781380727292} 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-20] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 70.90 μs 09:32:55,87 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1929317560] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,87 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:55,89 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:55,90 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 0 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 09:32:55,90 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4da372b0}, purged=MutableUnsignedLongSet{size=0}} 09:32:55,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1029539934] with name member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:32:55,91 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=0 09:32:55,91 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1029539934] 09:32:55,91 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,91 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 49.58 μs 09:32:55,91 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#-1029539934] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,91 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:55,93 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,93 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,93 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:55,94 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,94 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}]]] 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 1 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 2 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:55,95 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: no initial generation override, starting from 0 09:32:55,95 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: persisting new identifier ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Creating local shard: member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,95 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7471298f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e527aa1 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:55,95 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Setting last applied to 1 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: replication of member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@75d5fbb4 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: finished purging transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]}, enqueuedTicks=781380727292, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, sequence=1}} 09:32:55,96 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE}: purge completed 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fdd6351 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=DONE} 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:32:55,96 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@37ba099a purged transaction RemoteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, state=SUCCESSOR} 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,96 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:32:55,100 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@58c2c1ba 09:32:55,100 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: SaveSnapshotSuccess(SnapshotMetadata(member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, 0, 1714037575096, None)) 09:32:55,101 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037575095,0,0)) 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@57309d61 09:32:55,101 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4e527aa1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6ae7f3df 09:32:55,101 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} started 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1704520156] 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1704520156] 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop to akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,101 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: updated peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop on shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop whose actor address is Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1704520156] 09:32:55,102 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, persistent : true 09:32:55,102 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop with journal batch size 1 09:32:55,102 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] 09:32:55,102 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop#1704520156] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,102 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-522629921] 09:32:55,102 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2557/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1533707925 created and ready for shard:member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,102 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/shard-member-3:snapshot-read#-522629921] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,102 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-notifier#-1533707925] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,102 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493: creating a new transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:32:55,103 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:32:55,103 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager$GetLocalShards to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:55,103 AM [pool-34-thread-1] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493 readied transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-0 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:55,103 AM [pool-34-thread-1] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:55,103 AM [pool-34-thread-1] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@19da14b2 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-1-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,103 AM [pool-34-thread-1] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Peer address for peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,103 AM [pool-34-thread-1] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5bff22ed transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false}, enqueuedTicks=781397136810} 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Initializing shard [cars] 09:32:55,103 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized 09:32:55,103 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from null to Follower 09:32:55,104 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop , registered listener akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: allocating new ready transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct canCommit 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Validating transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 validated 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct preCommit 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Preparing transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 candidate ready 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - candidate rootPath: / 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: canCommit - messages: [] 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 prepared 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 initiating direct commit 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Starting commit for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 72 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Replicate message: identifier: member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: -1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]]] 09:32:55,104 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 submitted to persistence 09:32:55,105 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,105 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop from null to Follower 09:32:55,204 AM [awaitility-thread] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] 09:32:55,204 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Cluster state: [] 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Candidate in requestVote:RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Term 5 in "RequestVote [term=5, candidateId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=-1, lastLogTerm=-1]" message is greater than leader's term 4 - switching to Follower 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=5, votedFor=null 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader) :- Switching from behavior Leader to Follower, election term: 5 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-896: Closing 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: removing frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 09:32:55,205 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,205 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Leader to Follower 09:32:55,205 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=-1] 09:32:55,205 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,205 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Leader to Follower 09:32:55,205 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5bff22ed superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4533e3e9, splicing queue 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5bff22ed queue spliced 1 messages 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,206 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] 09:32:55,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:55,206 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Follower, leaderId=null, activeMember=true] 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Received ElectionTimeout - switching to Candidate 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Election: Candidate has following voting peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Starting new election term 6 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): Sending RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] to peer member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower) :- Switching from behavior Follower to Candidate, election term: 6 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: false, hasLeader : false 09:32:55,442 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is no longer the leader 09:32:55,443 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Follower to Candidate 09:32:55,443 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Follower to Candidate 09:32:55,443 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 0 - updating term 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,444 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Cluster state: [] 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Candidate in requestVote:RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] with higher term appears reachable, updating term. 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Term 6 in "RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4]" message is greater than follower's term 4 - updating term 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=null 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): In requestVote: RequestVote [term=6, candidateId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, lastLogIndex=2, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Set currentTerm=6, votedFor=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,444 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): requestVote returning: RequestVoteReply [term=6, voteGranted=true] 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=true], my term is 6 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=true], current voteCount: 1 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate): LastApplied index 1 is behind last index 2 - switching to PreLeader 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 6 09:32:55,445 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: became leader with frontend state for [member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop] 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persist data SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,446 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from Candidate to PreLeader 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Replicate message: identifier: null, logIndex: 3, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from Candidate to PreLeader 09:32:55,446 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: false, followerNextIndex: 1, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is initialized - notifying 1 OnShardInitialized callbacks 09:32:55,446 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 09:32:55,446 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=3] 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: getLeaderAddress leaderId = member-1-shard-cars-testRaftCallbackDuringLeadershipDrop peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderPayloadVersion=12] 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:32:55,447 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done false 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 1, nextIndex: 2 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,447 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,448 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): The followers log is empty and the senders prevLogIndex is 0 09:32:55,448 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Follower is out-of-sync so sending negative reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=3] 09:32:55,448 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop Received follower initial sync status for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop status sync done true 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 2 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 09:32:55,448 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-2-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:55,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,449 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: -1, nextIndex: 0 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - received unsuccessful reply: AppendEntriesReply [term=6, success=false, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], leader snapshotIndex: -1, snapshotTerm: -1, replicatedToAllIndex: -1 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop appears to be behind the leader from the last snapshot - updated: matchIndex: -1, nextIndex: 0 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Not updating commit index to 2 - retrieved log entry with index 2, term 4 does not match the current term 6 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 2 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, size=72} 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Resuming commit of transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,449 AM [cluster-test-shard-dispatcher-13] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1: commit - successfulFromPrevious: [] 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=174} 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Setting last applied to 3 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log Simple{size=72} 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 1 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Request for Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} not found in inflight queue, checking pending queue 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader): Received ApplyState [identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}]] - lastApplied: 3, lastIndex: 3 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 3 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 0 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from -1 to 0 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 0 data Simple{size=174} 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418], coordinated=false}, enqueuedTicks=781397136810, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Election: Leader has following peers: [member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED} directCommit completed 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 0, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): sendAppendEntries: 0 is present for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=FLUSHED}: initiating purge 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:32:55,450 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Sending AppendEntries to follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7fea9419 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: -1 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [INFO] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 6 09:32:55,450 AM [cluster-test-shard-dispatcher-13] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (PreLeader)_slicer-id-899: Closing 09:32:55,450 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onLeadershipChange, isLeader: true, hasLeader : true 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 29.58 μs 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1f2b1fc0}, purged=MutableUnsignedLongSet{size=0}} 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop , received role change from PreLeader to Leader 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1404445480] with name member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1404445480] 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 1 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 0 to 1 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66} 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,451 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Purged transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop/member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher#1404445480] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,451 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 69.32 μs 09:32:55,451 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received role changed for member-1-shard-cars-testRaftCallbackDuringLeadershipDrop from PreLeader to Leader 09:32:55,451 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardInformation - Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop is ready - notifying 1 OnShardInitialized callbacks 09:32:55,452 AM [cluster-test-shard-dispatcher-13] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@271d7e4e] 09:32:55,452 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All Shards are ready - data store testRaftCallbackDuringLeadershipDrop is ready 09:32:55,452 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@271d7e4e] 09:32:55,452 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@46963ce4, attempting to connect 09:32:55,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): handleAppendEntriesReply - FollowerLogInformation for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop updated: matchIndex: 3, nextIndex: 4 09:32:55,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: fakeSnapshot purging log to 2 for term 6 09:32:55,452 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:32:55,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] 09:32:55,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] 09:32:55,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] 09:32:55,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: persist complete SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:32:55,452 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:32:55,452 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=1 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fdd6351 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@ebee9c8 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} previous state DONE 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@ebee9c8 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 for transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#531310418]} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=OPEN}: initiating purge 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} allocated sequence 0 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@79cff569 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3820266b transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418]}, enqueuedTicks=781744122223} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@71333493 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@4129680e 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@38fd5b24 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3e9f95ef, splicing queue 09:32:55,453 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@38fd5b24 queue spliced 0 messages 09:32:55,454 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 813.4 μs 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,454 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=3, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}], SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,454 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Number of entries to be appended = 4 09:32:55,454 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,455 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 09:32:55,455 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=174}] is present: existingEntryTerm: 4 09:32:55,455 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-0-1, size=66}] is present: existingEntryTerm: 4 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=72}] is present: existingEntryTerm: 4 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): matchEntry SimpleReplicatedLogEntry [index=3, term=6, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] is present: existingEntryTerm: 6 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): After cleanup, lastIndex: 3, entries to be added from: 4 09:32:55,455 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Log size is now 4 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Commit index set to 3 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): handleAppendEntries returning : AppendEntriesReply [term=6, success=true, followerId=member-3-shard-cars-testRaftCallbackDuringLeadershipDrop, logLastIndex=3, logLastTerm=6, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 2 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 1 to 2 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 2 data Simple{size=72} 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying foreign transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, history=0, cookie=1} 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Committed transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Setting last applied to 3 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Moving last applied index from 2 to 3 09:32:55,455 AM [cluster-test-shard-dispatcher-13] [WARN] AbstractFrontendHistory - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: transaction member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1 not tracked in MutableUnsignedLongSet{span=[0..0], size=1}, but not present in active transactions 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Applying state for log index 3 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:32:55,455 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Persisting ApplyJournalEntries with index=3 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-2-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 38.69 μs 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: publishChanges: rootPath: / 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Elapsed time for generation of change events for member-3-shard-cars-testRaftCallbackDuringLeadershipDrop-DTCL-publisher: 7.830 μs 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#531310418]}, enqueuedTicks=781744122223, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, sequence=0}} 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE}: purge completed 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@ebee9c8 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=DONE} 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 09:32:55,455 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@2fdd6351 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testRaftCallbackDuringLeadershipDrop-fe-0-txn-1-1, state=SUCCESSOR} 09:32:55,507 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:55,507 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,507 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:32:55,507 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:32:55,508 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-3-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,508 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:32:55,508 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:32:55,508 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:32:55,508 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:55,508 AM [main] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,508 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:55,508 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 09:32:55,508 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] MessageAssembler - member-3-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,509 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:32:55,509 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#2115150715] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-2-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop (Follower): Closing 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:32:55,509 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:32:55,509 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:55,510 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,510 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0} 09:32:55,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 09:32:55,510 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Issuing gracefulStop to shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,510 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Shutting down ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop - waiting on 1 shards 09:32:55,510 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#1494351166] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,510 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1253891138] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1253891138] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: onShutDown 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Initiating leader transfer 09:32:55,510 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderId=null, leaderPayloadVersion=12] 09:32:55,510 AM [cluster-test-shard-dispatcher-13] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2237b77a 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@2237b77a 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Attempting to transfer leadership 09:32:55,510 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-3-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:32:55,511 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testRaftCallbackDuringLeadershipDrop: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3820266b superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@735ef29, splicing queue 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3820266b queue spliced 0 messages 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] ClientActorBehavior - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: refreshing backend for shard 1 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:55,511 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] 09:32:55,511 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:55,511 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testRaftCallbackDuringLeadershipDrop, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:32:55,512 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,512 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-1-shard-cars-testRaftCallbackDuringLeadershipDrop#1783853658] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop/member-2-shard-cars-testRaftCallbackDuringLeadershipDrop] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,802 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Failed to transfer leadership in 292.3 ms 09:32:55,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: In unpauseLeader 09:32:55,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: leader transfer failed - sending PoisonPill 09:32:55,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader): Leadership transfer expired 09:32:55,803 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testRaftCallbackDuringLeadershipDrop 09:32:55,803 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop (Leader)_slicer-id-900: Closing 09:32:55,804 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testRaftCallbackDuringLeadershipDrop: Aborting 0 pending queued transactions 09:32:55,804 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testRaftCallbackDuringLeadershipDrop: All shards shutdown - sending PoisonPill to self 09:32:55,804 AM [cluster-test-shard-dispatcher-13] [INFO] ShardManager - Stopping ShardManager shard-manager-testRaftCallbackDuringLeadershipDrop 09:32:55,804 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,804 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: shutting down 09:32:55,804 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-1-frontend-datastore-testRaftCallbackDuringLeadershipDrop: Closing 09:32:55,805 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testRaftCallbackDuringLeadershipDrop#-1450885430] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:55,805 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:55,805 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:55,809 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:55,809 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:55,809 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:55,809 AM [cluster-test-akka.actor.default-dispatcher-26] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:55,810 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:55,828 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:55,828 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:56,820 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4050052283174432206] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:56,840 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [841458959469681748] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:56,844 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:56,845 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:56,845 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:56.847] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:56,854 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:56,854 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:56,855 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:56.856] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:56,876 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:56,880 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-2131598797024771322] 09:32:56,880 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:56,881 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:56,881 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:56,895 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:56,895 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:56,897 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:56,897 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:56,897 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:56,899 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:56,905 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2817527049930106661] 09:32:56,905 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:56,911 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:56,911 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:56,913 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:56,913 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:56,923 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:56,933 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2496963999154048063] 09:32:56,933 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:56,933 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:56,933 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:56,937 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:56,937 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:56,939 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:56,941 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:56,942 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 09:32:56,947 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 09:32:56,950 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:56,950 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790934682] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,952 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 09:32:56,952 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,952 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:56,952 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,953 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:56,957 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 09:32:56,957 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:32:56,957 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@3b931742 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12bffcf9 09:32:56,957 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7b5423f2 09:32:56,958 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionChainWithSingleShard, 0, 1714037576957, None)) 09:32:56,958 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037576956,0,0)) 09:32:56,958 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@b463331 09:32:56,958 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@12bffcf9 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3bd08d6e 09:32:56,958 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 09:32:56,959 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:56,959 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:56,959 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,960 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,960 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-1-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219] 09:32:56,960 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:32:56,960 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionChainWithSingleShard, persistent : true 09:32:56,960 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,961 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 09:32:56,961 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-649949881] 09:32:56,961 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#901563709] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,961 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-notifier#901563709 created and ready for shard:member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,962 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionChainWithSingleShard 09:32:56,962 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/shard-member-1:snapshot-read#-649949881] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,962 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:56,962 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:56,963 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:56,963 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:32:56,963 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 09:32:56,963 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionChainWithSingleShard is initialized 09:32:56,963 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 09:32:56,963 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 09:32:56,963 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:32:56,964 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from null to Follower 09:32:56,965 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:32:56,966 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionChainWithSingleShard 09:32:56,966 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,968 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: no initial generation override, starting from 0 09:32:56,968 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:32:56,968 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@70c99aef to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6107befd 09:32:56,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionChainWithSingleShard 09:32:56,968 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Creating local shard: member-2-shard-cars-testTransactionChainWithSingleShard 09:32:56,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:32:56,968 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,969 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@65382605 09:32:56,969 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionChainWithSingleShard, 0, 1714037576968, None)) 09:32:56,969 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037576967,0,0)) 09:32:56,969 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@6107befd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@4dfcee51 09:32:56,969 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} started 09:32:56,969 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@5c484304 09:32:56,969 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:32:56,969 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionChainWithSingleShard 09:32:56,970 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:56,970 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: updated peer member-1-shard-cars-testTransactionChainWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard on shard member-2-shard-cars-testTransactionChainWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-2079517228] 09:32:56,970 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionChainWithSingleShard, persistent : true 09:32:56,970 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard#-2079517228] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,970 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#898629049 created and ready for shard:member-2-shard-cars-testTransactionChainWithSingleShard 09:32:56,970 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1804709590] 09:32:56,970 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-notifier#898629049] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,970 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionChainWithSingleShard with journal batch size 1 09:32:56,970 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/shard-member-2:snapshot-read#-1804709590] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:56,971 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:32:56,971 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:32:56,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Initializing shard [cars] 09:32:56,971 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionChainWithSingleShard is initialized 09:32:56,971 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:56,971 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:32:56,971 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:56,971 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , received role change from null to Follower 09:32:56,971 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionChainWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard 09:32:56,971 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-2-shard-cars-testTransactionChainWithSingleShard from null to Follower 09:32:57,20 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790934682] 09:32:57,20 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: findLocalShard : cars 09:32:57,20 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219] 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 1 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionChainWithSingleShard is no longer the leader 09:32:57,245 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Follower to Candidate 09:32:57,245 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Follower to Candidate 09:32:57,504 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:32:57,504 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:32:57,504 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 2 09:32:57,566 AM [cluster-test-shard-dispatcher-29] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:32:57,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:32:57,764 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:32:57,764 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 3 09:32:57,916 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:32:57,916 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:57,916 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionChainWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionChainWithSingleShard/member-3-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:57,924 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionChainWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard to akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard 09:32:57,941 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:32:58,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Received ElectionTimeout 09:32:58,64 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:32:58,64 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Starting new election term 4 09:32:58,65 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Cluster state: [] 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=null 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionChainWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionChainWithSingleShard 09:32:58,66 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionChainWithSingleShard] 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [INFO] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:32:58,67 AM [cluster-test-shard-dispatcher-30] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionChainWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: became leader with frontend state for [] 09:32:58,68 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionChainWithSingleShard , received role change from Candidate to Leader 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=12] 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received role changed for member-1-shard-cars-testTransactionChainWithSingleShard from Candidate to Leader 09:32:58,68 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 09:32:58,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard: Closing 09:32:58,68 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testTransactionChainWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard 09:32:58,69 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done false 09:32:58,69 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionChainWithSingleShard, leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderPayloadVersion=12] 09:32:58,69 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:58,69 AM [cluster-test-shard-dispatcher-29] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:58,69 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: All Shards are ready - data store testTransactionChainWithSingleShard is ready 09:32:58,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:32:58,69 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:32:58,76 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f 09:32:58,76 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f changed state from IDLE to TX_OPEN 09:32:58,76 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 09:32:58,76 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:32:58,76 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:32:58,76 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] 09:32:58,76 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=0} shard 1 09:32:58,76 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:32:58,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:32:58,76 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:32:58,76 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:58,76 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=12] 09:32:58,77 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard, primaryVersion=12] 09:32:58,77 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@7f521908 09:32:58,77 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f changed state from TX_OPEN to IDLE 09:32:58,77 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:32:58,77 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 09:32:58,77 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 09:32:58,77 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@5df4cbdf 09:32:58,79 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@1962a6db, attempting to connect 09:32:58,80 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} is not yet known 09:32:58,80 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, nanosAgo=9410, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:32:58,81 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1a487788 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b0d2cfe 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state SEALED 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} previous state OPEN 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784370058887} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b0d2cfe 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@72511f07 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY}, enqueuedTicks=784370396543} 09:32:58,81 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY}} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 389 bytes, total slices to send: 1 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b0d2cfe 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} allocated sequence 0 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@3a9940f0 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=784370650167} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@a33efd6 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@211ad3b7 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@a2b074a, splicing queue 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@211ad3b7 queue spliced 0 messages 09:32:58,82 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionChainWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.161 ms 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@28fa006c 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 ready 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}]} 09:32:58,83 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 to ready 09:32:58,84 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 09:32:58,84 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,84 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Allocated read-only transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE}}}] 09:32:58,84 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,84 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 09:32:58,84 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message size: 331 bytes, total slices to send: 1 09:32:58,84 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message does not need to be sliced - sending original message 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY}, enqueuedTicks=784370396543, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0}} 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=1, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0} 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=784370650167, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0}} 09:32:58,85 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=0} 09:32:58,85 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f changed state from IDLE to TX_OPEN 09:32:58,85 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:32:58,85 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 0 09:32:58,85 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@b4dc081 09:32:58,85 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=784378841126} 09:32:58,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Reusing a chained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,86 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 snapshot MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}] 09:32:58,86 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:58,86 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message size: 331 bytes, total slices to send: 1 09:32:58,86 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message does not need to be sliced - sending original message 09:32:58,87 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:58,87 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=784378841126, sessionId=0, txSequence=3} with Envelope{sessionId=0, txSequence=3, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0}} 09:32:58,87 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=0} 09:32:58,87 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 1 09:32:58,87 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 2 09:32:58,87 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@172ac29d 09:32:58,87 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2}, enqueuedTicks=784380907272} 09:32:58,87 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=4, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2}} 09:32:58,87 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 459 bytes, total slices to send: 1 09:32:58,87 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,87 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1324/0x000000080135cf70@30da5a34 09:32:58,88 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=784381171906} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2}, enqueuedTicks=784380907272, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1}} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=1} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ExistsTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}]}, enqueuedTicks=784381171906, sessionId=0, txSequence=5} with Envelope{sessionId=0, txSequence=5, message=ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true}} 09:32:58,89 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Exists request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car/car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=optima}] completed with ExistsTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=2, exists=true} 09:32:58,90 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 3 09:32:58,90 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@534f974f 09:32:58,90 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=784383358322} 09:32:58,90 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 09:32:58,90 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message size: 369 bytes, total slices to send: 1 09:32:58,90 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message does not need to be sliced - sending original message 09:32:58,92 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 09:32:58,93 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=false, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=784383358322, sessionId=0, txSequence=6} with Envelope{sessionId=0, txSequence=6, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3}} 09:32:58,93 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=3} 09:32:58,93 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} allocated sequence 4 09:32:58,93 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 09:32:58,93 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@4edde35f 09:32:58,93 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}, enqueuedTicks=784386576018} 09:32:58,93 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=7, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}} 09:32:58,93 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 285 bytes, total slices to send: 1 09:32:58,93 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 is already in state READY (READY) 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct canCommit 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 validated 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct preCommit 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 candidate ready 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - candidate rootPath: / 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: canCommit - messages: [] 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: preCommit - successfulFromPrevious: [] 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 prepared 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 initiating direct commit 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 174 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:32:58,94 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,95 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 09:32:58,95 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 submitted to persistence 09:32:58,95 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}], SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}]]] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=174} 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 2 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=174}] 09:32:58,97 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard Received follower initial sync status for member-2-shard-cars-testTransactionChainWithSingleShard status sync done true 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 1, nextIndex: 2 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: -1 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 0 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} completed, invoking org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled$$Lambda$2178/0x000000080169b4b8@1dcf6c05 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] LeaderFrontendState - member-1-shard-cars-testTransactionChainWithSingleShard: persisted history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@737eb3d5}, purged=MutableUnsignedLongSet{size=0}} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 1 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1: commit - successfulFromPrevious: [] 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=WRITE, modificationType=WRITE}}}]} 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#616015473] with name member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 09:32:58,98 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#616015473] 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:32:58,98 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 09:32:58,98 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:32:58,99 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 67.35 μs 09:32:58,99 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#616015473] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 1 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): -1, lastIndex(): 1 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 0 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from -1 to 0 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 0 data CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65} 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionChainWithSingleShard: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5d58d793}, purged=MutableUnsignedLongSet{size=0}} 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Created local history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 1 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 0 to 1 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 1 data Simple{size=174} 09:32:58,100 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,101 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,101 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionChainWithSingleShard: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1873886621] with name member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher 09:32:58,101 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 0 for term 4 09:32:58,101 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=1 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1873886621] 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543]} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784370058887, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0}} 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientHistory - Create history response LocalHistorySuccess{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0} 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}, enqueuedTicks=784386576018, sessionId=0, txSequence=7} with Envelope{sessionId=0, txSequence=7, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1}} 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 71.27 μs 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} directCommit completed 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher#1873886621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED}: initiating purge 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 09:32:58,101 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@50d0424f 09:32:58,101 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7a45583f 09:32:58,101 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:58,101 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY}, enqueuedTicks=784395080233} 09:32:58,101 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=8, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY}} 09:32:58,101 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 473 bytes, total slices to send: 1 09:32:58,101 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,102 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f changed state from TX_OPEN to IDLE 09:32:58,102 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784395025962} 09:32:58,102 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f readied transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:32:58,102 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} allocated sequence 5 09:32:58,102 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6fa292b4 09:32:58,102 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}, enqueuedTicks=784395391988} 09:32:58,102 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, message: Envelope{sessionId=0, txSequence=a, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}} 09:32:58,102 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 286 bytes, total slices to send: 1 09:32:58,102 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: readying transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 ready 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeTransactionChain - Committing transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE}}}}}}}]} 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: transitioned member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 to ready 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 is already in state READY (READY) 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct canCommit 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Validating transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 validated 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct preCommit 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Preparing transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 candidate ready 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - candidate rootPath: / 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: canCommit - messages: [] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: preCommit - successfulFromPrevious: [] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 prepared 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 initiating direct commit 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Starting commit for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 230 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 2, leaderLastIndex: 3, leaderSnapShotIndex: 0 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,103 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 09:32:58,103 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY}, enqueuedTicks=784395080233, sessionId=0, txSequence=8} with Envelope{sessionId=0, txSequence=8, message=ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4}} 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 submitted to persistence 09:32:58,103 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=2, protocol=READY} completed with ModifyTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=4} 09:32:58,103 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}], SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}]]] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Number of entries to be appended = 2 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Append entry to log Simple{size=230} 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Log size is now 3 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionChainWithSingleShard, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66}] 09:32:58,104 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=Simple{size=230}] 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionChainWithSingleShard updated: matchIndex: 3, nextIndex: 4 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 1 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 2 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: replication of member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@2d674ea2 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: finished purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Setting last applied to 3 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 09:32:58,104 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Resuming commit of transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1: commit - successfulFromPrevious: [] 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] ChainedCommitCohort - Committed transaction ReadWriteShardDataTreeTransaction{id=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, closed=true, snapshot=MutableDataTree [modification=ModifiedNode{identifier=(urn:ietf:params:xml:ns:netconf:base:1.0)data, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, operation=TOUCH, modificationType=SUBTREE_MODIFIED, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car, operation=MERGE, modificationType=WRITE, childModification={(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}]=ModifiedNode{identifier=(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)car[{(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)name=sportage}], operation=MERGE, modificationType=WRITE}}}}}}}]} 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:32:58,105 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:32:58,105 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 09:32:58,105 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-1-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 40.45 μs 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Commit index set to 3 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 1, lastIndex(): 3 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 2 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 1 to 2 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 2 data PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, size=66} 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Purged transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Setting last applied to 3 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Moving last applied index from 2 to 3 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Applying state for log index 3 data Simple{size=230} 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionChainWithSingleShard: Applying foreign transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionChainWithSingleShard: Committed transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: fakeSnapshot purging log to 2 for term 4 09:32:58,108 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Persisting ApplyJournalEntries with index=3 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionChainWithSingleShard: publishChanges: rootPath: / 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionChainWithSingleShard: Elapsed time for generation of change events for member-2-shard-cars-testTransactionChainWithSingleShard-DTCL-publisher: 35.21 μs 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543]} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784395025962, sessionId=0, txSequence=9} with Envelope{sessionId=0, txSequence=9, message=TransactionPurgeResponse{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2}} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE}: purge completed 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b0d2cfe purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=SIMPLE}, enqueuedTicks=784395391988, sessionId=0, txSequence=10} with Envelope{sessionId=0, txSequence=a, message=TransactionCommitSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=5}} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED} directCommit completed 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=FLUSHED}: initiating purge 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} allocated sequence 6 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#168780543]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@6f1d55fe 09:32:58,109 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-0-0 09:32:58,109 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-0 09:32:58,109 AM [main] [DEBUG] ClientBackedTransactionChain - Aborting recorded transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:32:58,109 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@1a487788 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} 09:32:58,109 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, sequence=6, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784402575529} 09:32:58,109 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} allocated sequence 1 09:32:58,109 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@559822bc 09:32:58,109 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionChainWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, cookie=1} 09:32:58,109 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} allocated sequence 1 09:32:58,109 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@48aaf81e 09:32:58,109 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT}, enqueuedTicks=784402748152} 09:32:58,109 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, message: Envelope{sessionId=0, txSequence=c, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT}} 09:32:58,109 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 286 bytes, total slices to send: 1 09:32:58,109 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:58,109 AM [main] [WARN] ClientBackedTransactionChain - Aborted unclosed transactions [member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-0] java.lang.Throwable: at at org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain.close(ClientBackedTransactionChain.java:76) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.testTransactionChainWithSingleShard(DistributedDataStoreRemotingIntegrationTest.java:581) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) 09:32:58,109 AM [main] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@5b0d2cfe invoking destroy 09:32:58,109 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=DestroyLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543]}, enqueuedTicks=784403195740} 09:32:58,109 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@31fb8f5f changed state from IDLE to CLOSED 09:32:58,109 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@6fa3e620: creating a new snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 09:32:58,110 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=0, cookie=0} shard 1 09:32:58,110 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0 09:32:58,110 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:32:58,110 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@bd68912 09:32:58,110 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=784403409894} 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: purging transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: aborting transaction member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, logIndex: 5, payload: class org.opendaylight.controller.cluster.datastore.persisted.AbortTransactionPayload, isSendImmediate: false 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardDataTreeTransactionChain - Closing chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Persist data SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 6, payload: class org.opendaylight.controller.cluster.datastore.persisted.CloseLocalHistoryPayload, isSendImmediate: false 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionChainWithSingleShard: allocating new open snapshot member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1 09:32:58,110 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}] 09:32:58,110 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:58,111 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}] 09:32:58,111 AM [cluster-test-shard-dispatcher-30] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: persist complete SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}] 09:32:58,111 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message size: 369 bytes, total slices to send: 1 09:32:58,111 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard_slicer-id-902: Message does not need to be sliced - sending original message 09:32:58,111 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:58,111 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#168780543], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car}, enqueuedTicks=784403409894, sessionId=0, txSequence=14} with Envelope{sessionId=0, txSequence=e, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0}} 09:32:58,111 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=0} 09:32:58,112 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:32:58,112 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,112 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 09:32:58,112 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: onShutDown 09:32:58,112 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,112 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionChainWithSingleShard (Follower): Closing 09:32:58,112 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-2-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 09:32:58,112 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 09:32:58,113 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 09:32:58,113 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:32:58,113 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionChainWithSingleShard, generation=0} 09:32:58,113 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionChainWithSingleShard: shutting down 09:32:58,113 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:32:58,113 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,113 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#1852752750] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,113 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#168780543] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#168780543] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,113 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testTransactionChainWithSingleShard 09:32:58,114 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionChainWithSingleShard - waiting on 1 shards 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: onShutDown 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Initiating leader transfer 09:32:58,114 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionChainWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionChainWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:32:58,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:32:58,114 AM [cluster-test-shard-dispatcher-30] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b3f1283 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1} 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionChainWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@b3f1283 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Attempting to transfer leadership 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionChainWithSingleShard: active: true, followerNextIndex: 4, leaderLastIndex: 6, leaderSnapShotIndex: 2 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionChainWithSingleShard 09:32:58,114 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionChainWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testTransactionChainWithSingleShard, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-2-1, size=67}], SimpleReplicatedLogEntry [index=5, term=4, payload=AbortTransactionPayload{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-chn-1-txn-1-1, size=67}], SimpleReplicatedLogEntry [index=6, term=4, payload=CloseLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionChainWithSingleShard, generation=0}, history=1, cookie=1}, size=65}]]] 09:32:58,114 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,115 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionChainWithSingleShard/member-1-shard-cars-testTransactionChainWithSingleShard#850769219] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard/member-2-shard-cars-testTransactionChainWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,434 AM [cluster-test-shard-dispatcher-27] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionChainWithSingleShard: Failed to transfer leadership in 320.4 ms 09:32:58,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: In unpauseLeader 09:32:58,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: leader transfer failed - sending PoisonPill 09:32:58,434 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard (Leader): Leadership transfer expired 09:32:58,435 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionChainWithSingleShard 09:32:58,435 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionChainWithSingleShard (Leader)_slicer-id-905: Closing 09:32:58,435 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-1-shard-cars-testTransactionChainWithSingleShard: Aborting 0 pending queued transactions 09:32:58,435 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionChainWithSingleShard: All shards shutdown - sending PoisonPill to self 09:32:58,436 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionChainWithSingleShard 09:32:58,436 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:32:58,436 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionChainWithSingleShard: shutting down 09:32:58,436 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionChainWithSingleShard: Closing 09:32:58,436 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790934682] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionChainWithSingleShard#-790934682] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:32:58,437 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:58,437 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:58,443 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:58,443 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:58,443 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:58,443 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:58,456 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:58,456 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:59,450 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-2131598797024771322] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:59,470 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2817527049930106661] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:32:59,478 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:59,478 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:59,481 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:59.483] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:59,489 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:59,489 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:59,491 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:32:59.492] [cluster-test-akka.actor.internal-dispatcher-9] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:59,511 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:59,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-521205818752920172] 09:32:59,516 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:32:59,522 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:32:59,522 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:32:59,522 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:59,522 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:59,528 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:32:59,528 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:32:59,528 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:32:59,530 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:32:59,534 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-3603301059618055295] 09:32:59,534 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:32:59,536 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:59,536 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:32:59,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:59,543 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:59,548 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@40b863c0 aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} 09:32:59,548 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} allocated sequence 1 09:32:59,548 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@6cb5fe85 09:32:59,548 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@3e9887ba transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT}, enqueuedTicks=785841795819} 09:32:59,548 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: slice: identifier: member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=f, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#168780543], modifications=0, protocol=ABORT}} 09:32:59,548 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message size: 284 bytes, total slices to send: 1 09:32:59,548 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionChainWithSingleShard_slicer-id-903: Message does not need to be sliced - sending original message 09:32:59,548 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionChainWithSingleShard-fe-0-txn-0-0} 09:32:59,558 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:32:59,580 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [5739938876660409482] 09:32:59,581 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:32:59,582 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:32:59,582 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:32:59,594 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:59,594 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:32:59,596 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:32:59,596 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:32:59,603 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:32:59,607 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:59,608 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:32:59,610 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:32:59,611 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], message stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:32:59,613 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2558], control stream] Upstream failed, cause: StreamTcpException: Tcp command [Connect(127.0.0.1/:2558,None,List(),Some(5000 milliseconds),true)] failed because of java.net.ConnectException: Connection refused 09:32:59,617 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:32:59,617 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:00,634 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:00.635] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:00,651 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:00,651 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:00,652 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:00.653] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:00,668 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:00,673 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-9093008670629262959] 09:33:00,673 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:00,674 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:00,674 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:00,683 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:00,683 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:00,683 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:00,683 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:00,683 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:00,687 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:00,692 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-8273030185344463036] 09:33:00,692 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:00,693 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:00,693 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:00,699 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:00,699 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:00,707 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:00,720 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [4347331077833236696] 09:33:00,720 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:00,721 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:00,721 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:00,724 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:00,724 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:00,729 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:00,730 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:00,731 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:00,732 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:00,733 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 09:33:00,738 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:00,739 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,743 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:00,743 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,744 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:00,744 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:00,744 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,746 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:00,748 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 09:33:00,748 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:33:00,748 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@528de592 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7eb27c89 09:33:00,748 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@7a8df787 09:33:00,748 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1714037580748, None)) 09:33:00,749 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037580747,0,0)) 09:33:00,749 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@7eb27c89 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@6341c790 09:33:00,749 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@79d55f56 09:33:00,754 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 09:33:00,755 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:00,755 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,755 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:00,755 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,755 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,755 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062] 09:33:00,755 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 09:33:00,756 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 09:33:00,756 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-979441884 created and ready for shard:member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,756 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,756 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#1540268133] 09:33:00,756 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-979441884] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,756 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-1:snapshot-read#1540268133] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,756 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:00,756 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:00,756 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:00,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 09:33:00,757 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 09:33:00,757 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:00,757 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:33:00,757 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 09:33:00,757 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 09:33:00,757 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 09:33:00,758 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testForwardedReadyTransactionForwardedToLeader 09:33:00,760 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,761 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:00,766 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: no initial generation override, starting from 0 09:33:00,766 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:00,766 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:33:00,766 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Creating local shard: member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,766 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@325a3072 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4f5d4be1 09:33:00,766 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@723ac7eb 09:33:00,766 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:00,766 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,766 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, 0, 1714037580766, None)) 09:33:00,767 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037580765,0,0)) 09:33:00,767 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@7532059f 09:33:00,767 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4f5d4be1 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@3f6dc414 09:33:00,767 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} started 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: updated peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader on shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] 09:33:00,768 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, persistent : true 09:33:00,768 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,768 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1228964275] 09:33:00,769 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1048193249] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,769 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-notifier#-1048193249 created and ready for shard:member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,769 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with journal batch size 1 09:33:00,769 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/shard-member-2:snapshot-read#1228964275] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:00,770 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Initializing shard [cars] 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:00,770 AM [cluster-test-shard-dispatcher-26] [DEBUG] ShardInformation - Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is initialized 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:33:00,770 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from null to Follower 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:00,770 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader , registered listener akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader 09:33:00,770 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader from null to Follower 09:33:00,818 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] 09:33:00,818 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:33:00,818 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062] 09:33:01,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:01,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Received ElectionTimeout - switching to Candidate 09:33:01,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 09:33:01,26 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,26 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 1 09:33:01,27 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:01,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 09:33:01,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:01,27 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader is no longer the leader 09:33:01,27 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Follower to Candidate 09:33:01,27 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Follower to Candidate 09:33:01,314 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:01,346 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:33:01,346 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,346 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 2 09:33:01,636 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:33:01,636 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,636 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 3 09:33:01,686 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:01,686 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,687 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-3-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,695 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader to akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,736 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:01,896 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Received ElectionTimeout 09:33:01,896 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,896 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Starting new election term 4 09:33:01,896 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Cluster state: [] 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=null 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,898 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:01,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:01,903 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: became leader with frontend state for [] 09:33:01,904 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader , received role change from Candidate to Leader 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=12] 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received role changed for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader from Candidate to Leader 09:33:01,904 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 09:33:01,905 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Closing 09:33:01,905 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,905 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderPayloadVersion=12] 09:33:01,905 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:01,905 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:01,905 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All Shards are ready - data store testForwardedReadyTransactionForwardedToLeader is ready 09:33:01,905 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done false 09:33:01,906 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: -1, nextIndex: 0 09:33:01,906 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:01,975 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] 09:33:01,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:33:01,975 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] 09:33:01,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,975 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] 09:33:01,975 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: findLocalShard : cars 09:33:01,975 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] 09:33:01,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,987 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 09:33:01,989 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: encountered ask-based client ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, disabling transaction tracking 09:33:01,989 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:33:01,989 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:33:01,989 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.DisableTrackingPayload, isSendImmediate: true 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 3 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-127-0 of 3 operations, client version 12 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-127-0 ready 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 validated 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-127-0: success 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,990 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 candidate ready 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: canCommit - candidate rootPath: / 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: canCommit - messages: [] 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: preCommit - successfulFromPrevious: [] 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 prepared 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 197 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-127-0, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,991 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 submitted to persistence 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}]]] 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 1 09:33:01,991 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:33:01,992 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37}] 09:33:01,992 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader Received follower initial sync status for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader status sync done true 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 0, nextIndex: 1 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 0 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:33:01,992 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader state Disabled{clientId=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, nanosAgo=3150315} is already disabled 09:33:01,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:33:01,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:01,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:33:01,993 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}]]] 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=197} 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 0 09:33:01,993 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 1 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 0 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from -1 to 0 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 0 data DisableTrackingPayload{identifier=ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0}, size=37} 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disabling tracking of ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendMetadata - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: disableTracking ClientIdentifier{frontend=member-1-frontend-ShardTransactionTest, generation=0} does not match any client, pre-disabling client. 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=Simple{size=197}] 09:33:01,994 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=0 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 1, nextIndex: 2 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 1 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-127-0: commit - successfulFromPrevious: [] 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 changing state from COMMIT_PENDING to COMMITTED 09:33:01,994 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-127-0 committed as 0, sending response to Actor[akka://cluster-test@127.0.0.1:2559/system/testProbe-651#-1651992766] 09:33:01,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1582628885] with name member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 09:33:01,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:33:01,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:33:01,995 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:33:01,995 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1582628885] 09:33:01,995 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:33:01,995 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1582628885] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:01,995 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 67.23 μs 09:33:01,997 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@283b9d04: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 09:33:01,997 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 09:33:01,997 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:01,997 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:01,997 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] 09:33:01,997 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=0} shard 1 09:33:01,997 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-0 09:33:01,997 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:01,997 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1544782627], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@5ba3daa7 09:33:01,997 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Commit index set to 1 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Setting last applied to 1 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 0 to 1 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 1 data Simple{size=197} 09:33:01,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying foreign transaction member-1-ShardTransactionTest-fe-0-txn-127-0 09:33:01,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1915036168] with name member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher 09:33:01,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 0 for term 4 09:33:01,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=1 09:33:01,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1915036168] 09:33:01,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:01,998 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@53c82e43] 09:33:01,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:33:01,998 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher#1915036168] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:01,998 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@53c82e43] 09:33:01,998 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 114.6 μs 09:33:01,999 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6ea47c8c, attempting to connect 09:33:01,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} is not yet known 09:33:01,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, nanosAgo=9180, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:33:01,999 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b060ff4 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@10cbe577 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@10cbe577 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1544782627], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car} to successor LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@283b9d04 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@21915152 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3f4bb0fb superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@70e41a38, splicing queue 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@3f4bb0fb queue spliced 0 messages 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 355.7 μs 09:33:02,0 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars/car completed with ReadTransactionSuccess{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0} 09:33:02,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: handleForwardedReadyTransaction for member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: getLeaderAddress leaderId = member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,0 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Forwarding ForwardedReadyTransaction to leader ActorSelection[Anchor(akka://cluster-test@127.0.0.1:2558/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader)] 09:33:02,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying 1 batched modifications for Tx member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Readying Tx member-1-ShardTransactionTest-fe-0-txn-128-0 of 1 operations, client version 12 09:33:02,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: readying transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,1 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: transaction member-1-ShardTransactionTest-fe-0-txn-128-0 ready 09:33:02,2 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 canCommit 09:33:02,2 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: cohort actor ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062)] resolved 09:33:02,2 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: successfully resolved all cohort actors 09:33:02,2 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 finishCanCommit 09:33:02,2 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: sending CanCommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-128-0, version=12] to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062)] 09:33:02,2 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CanCommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062)] 09:33:02,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Can committing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Validating transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 validated 09:33:02,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:02,2 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: canCommit for member-1-ShardTransactionTest-fe-0-txn-128-0: success 09:33:02,3 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: received CanCommitTransactionReply [canCommit=true, version=12] 09:33:02,3 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit returning result: true 09:33:02,3 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 commit 09:33:02,3 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0 finish commit 09:33:02,3 AM [main] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: Sending CommitTransaction [transactionId=member-1-ShardTransactionTest-fe-0-txn-128-0, version=12] to cohort ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062)] 09:33:02,3 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.CommitTransaction to ActorSelection[Anchor(akka://cluster-test/), Path(/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062)] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Committing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Preparing transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 candidate ready 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit - candidate rootPath: / 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: canCommit - messages: [] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: preCommit - successfulFromPrevious: [] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 prepared 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Finishing commit for transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Starting commit for transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 208 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Replicate message: identifier: member-1-ShardTransactionTest-fe-0-txn-128-0, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 0 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): sendAppendEntries: 2 is present for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 submitted to persistence 09:33:02,3 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, prevLogIndex=1, prevLogTerm=4, leaderCommit=1, replicatedToAllIndex=0, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}]]] 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Number of entries to be appended = 1 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): After cleanup, lastIndex: 1, entries to be added from: 0 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Append entry to log Simple{size=208} 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Log size is now 2 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:02,4 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=208}] 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader updated: matchIndex: 2, nextIndex: 3 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 1 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Setting last applied to 2 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Moving last applied index from 1 to 2 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Applying state for log index 2 data Simple{size=208} 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Resuming commit of transaction member-1-ShardTransactionTest-fe-0-txn-128-0 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-ShardTransactionTest-fe-0-txn-128-0: commit - successfulFromPrevious: [] 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 changing state from COMMIT_PENDING to COMMITTED 09:33:02,4 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Transaction member-1-ShardTransactionTest-fe-0-txn-128-0 committed as 0, sending response to Actor[akka://cluster-test/temp/_user_shardmanager-testForwardedReadyTransactionForwardedToLeader_member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader%23-1884521062$a] 09:33:02,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: fakeSnapshot purging log to 1 for term 4 09:33:02,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:02,5 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Persisting ApplyJournalEntries with index=2 09:33:02,5 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ThreePhaseCommitCohortProxy - Tx member-1-ShardTransactionTest-fe-0-txn-128-0: commit succeeded 09:33:02,5 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: publishChanges: rootPath: / 09:33:02,5 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Elapsed time for generation of change events for member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader-DTCL-publisher: 41.03 μs 09:33:02,5 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@283b9d04: creating a new snapshot member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 09:33:02,5 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1 for transaction member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-0 09:33:02,5 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:33:02,5 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,5 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 09:33:02,5 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 09:33:02,6 AM [cluster-test-shard-dispatcher-26] [INFO] TestShard - Stopping Shard member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,6 AM [cluster-test-shard-dispatcher-26] [DEBUG] MessageAssembler - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader (Follower): Closing 09:33:02,6 AM [cluster-test-shard-dispatcher-26] [DEBUG] TestShard - member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:33:02,6 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:33:02,6 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader#482246621] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,6 AM [cluster-test-shard-dispatcher-26] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:02,6 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:33:02,6 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0} 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Issuing gracefulStop to shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] MessageAssembler - member-2-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Shutting down ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader - waiting on 1 shards 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: onShutDown 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Initiating leader transfer 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@606f3201 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@606f3201 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Attempting to transfer leadership 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-282127727] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-282127727] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,7 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-277521187] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,7 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderId=null, leaderPayloadVersion=12] 09:33:02,7 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@244f7d06 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@74aa866d, splicing queue 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@244f7d06 queue spliced 0 messages 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: refreshing backend for shard 1 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:02,7 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] 09:33:02,8 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:02,8 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:33:02,8 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-2-shard-cars-testForwardedReadyTransactionForwardedToLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,256 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Failed to transfer leadership in 249.0 ms 09:33:02,256 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: In unpauseLeader 09:33:02,256 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: leader transfer failed - sending PoisonPill 09:33:02,256 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader): Leadership transfer expired 09:33:02,257 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader 09:33:02,257 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader (Leader)_slicer-id-910: Closing 09:33:02,257 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader: Aborting 0 pending queued transactions 09:33:02,257 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testForwardedReadyTransactionForwardedToLeader: All shards shutdown - sending PoisonPill to self 09:33:02,257 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Stopping ShardManager shard-manager-testForwardedReadyTransactionForwardedToLeader 09:33:02,258 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:33:02,258 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: shutting down 09:33:02,258 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageAssembler - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: Closing 09:33:02,258 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,258 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader#-226800275] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,258 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1544782627] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1544782627] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:02,259 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:02,259 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:02,263 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:02,263 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:02,263 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:02,264 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:02,265 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:02,277 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:02,277 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:03,276 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-9093008670629262959] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:03,296 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-8273030185344463036] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:03,297 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:03,297 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:03,298 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:03.298] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:03,309 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:03,309 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:03,310 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:03.311] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:03,325 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:03,331 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [4091770071642645903] 09:33:03,331 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:03,332 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:03,332 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:03,338 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:03,338 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:03,343 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:03,343 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:03,343 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:03,347 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:03,350 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2834138859200947380] 09:33:03,350 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:03,351 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:03,351 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:03,361 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:03,361 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:03,365 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:03,371 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [2777844013968708848] 09:33:03,371 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:03,374 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:03,374 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:03,374 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:03,374 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:03,380 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:03,381 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:03,383 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 09:33:03,386 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:03,388 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:33:03,391 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1828761360] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,394 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 09:33:03,394 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:33:03,395 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@36d440f6 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@454df61b 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@41727a6a 09:33:03,395 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:03,395 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1714037583395, None)) 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037583394,0,0)) 09:33:03,395 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@842385a 09:33:03,395 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@454df61b to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@74f011b 09:33:03,395 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 09:33:03,396 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:03,396 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,396 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:03,396 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,396 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,396 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] 09:33:03,396 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 09:33:03,397 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1055924427 created and ready for shard:member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1404179136] 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-1055924427] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-1:snapshot-read#1404179136] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,397 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:03,397 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:03,399 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testChainedTransactionFailureWithSingleShard 09:33:03,401 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:03,401 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:03,401 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 09:33:03,401 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 09:33:03,402 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:33:03,402 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:33:03,402 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 09:33:03,402 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 09:33:03,402 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 09:33:03,405 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:33:03,405 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,408 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: no initial generation override, starting from 0 09:33:03,408 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:33:03,408 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@335a0f32 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@552800a 09:33:03,408 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testChainedTransactionFailureWithSingleShard 09:33:03,408 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@43a34824 09:33:03,409 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Creating local shard: member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,409 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:03,409 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, 0, 1714037583409, None)) 09:33:03,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037583408,0,0)) 09:33:03,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@552800a to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@737d713d 09:33:03,409 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} started 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@297bdb56 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: updated peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard on shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard whose actor address is Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-951860764] 09:33:03,410 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testChainedTransactionFailureWithSingleShard, persistent : true 09:33:03,410 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard#-951860764] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,410 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-342326511 created and ready for shard:member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,410 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard-notifier#-342326511] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,410 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1927371934] 09:33:03,410 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard/shard-member-2:snapshot-read#-1927371934] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:03,411 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testChainedTransactionFailureWithSingleShard with journal batch size 1 09:33:03,411 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:03,411 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:03,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Initializing shard [cars] 09:33:03,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is initialized 09:33:03,412 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:33:03,412 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:33:03,412 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from null to Follower 09:33:03,412 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,412 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testChainedTransactionFailureWithSingleShard , registered listener akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard 09:33:03,412 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-2-shard-cars-testChainedTransactionFailureWithSingleShard from null to Follower 09:33:03,417 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:03,418 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:03,460 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1828761360] 09:33:03,460 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: findLocalShard : cars 09:33:03,460 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Received ElectionTimeout - switching to Candidate 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=1, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 1 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageAssembler - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: false, hasLeader : false 09:33:03,651 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard is no longer the leader 09:33:03,652 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Follower to Candidate 09:33:03,652 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Follower to Candidate 09:33:03,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:33:03,961 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=2, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:03,961 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 2 09:33:03,965 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:04,240 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:33:04,240 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=3, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,240 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 3 09:33:04,351 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:04,351 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,352 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,352 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-3-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,359 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard to akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Received ElectionTimeout 09:33:04,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,541 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Starting new election term 4 09:33:04,541 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Cluster state: [] 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=null 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Set currentTerm=4, votedFor=member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,542 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:04,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:04,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:04,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Election: Leader has following peers: [member-2-shard-cars-testChainedTransactionFailureWithSingleShard] 09:33:04,543 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onLeadershipChange, isLeader: true, hasLeader : true 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: became leader with frontend state for [] 09:33:04,544 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testChainedTransactionFailureWithSingleShard , received role change from Candidate to Leader 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=12] 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received role changed for member-1-shard-cars-testChainedTransactionFailureWithSingleShard from Candidate to Leader 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 09:33:04,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Closing 09:33:04,544 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: getLeaderAddress leaderId = member-1-shard-cars-testChainedTransactionFailureWithSingleShard peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderPayloadVersion=12] 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All Shards are ready - data store testChainedTransactionFailureWithSingleShard is ready 09:33:04,544 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard Received follower initial sync status for member-2-shard-cars-testChainedTransactionFailureWithSingleShard status sync done false 09:33:04,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testChainedTransactionFailureWithSingleShard updated: matchIndex: -1, nextIndex: 0 09:33:04,545 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:04,566 AM [main] [DEBUG] AbstractDataStoreClientBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: creating a new local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ad0a3c5 09:33:04,567 AM [main] [DEBUG] AbstractDOMBroker - Transaction chain 0 created with listener Mock for DOMTransactionChainListener, hashCode: 516112126, backing store chains {CONFIGURATION=org.opendaylight.controller.cluster.databroker.ClientBackedTransactionChain@653a03e0} 09:33:04,567 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ad0a3c5 changed state from IDLE to TX_OPEN 09:33:04,567 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 09:33:04,567 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:04,567 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:04,567 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] 09:33:04,567 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:04,567 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} shard 1 09:33:04,567 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:33:04,567 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=12] 09:33:04,567 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 09:33:04,567 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:33:04,567 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2170/0x000000080169e630@4bc45290 09:33:04,567 AM [main] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ad0a3c5 changed state from TX_OPEN to IDLE 09:33:04,567 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ad0a3c5 readied transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-0 09:33:04,567 AM [main] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 is submitted for execution. 09:33:04,567 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} allocated sequence 1 09:33:04,567 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@48456118 09:33:04,568 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard, primaryVersion=12] 09:33:04,568 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6b62e7b7, attempting to connect 09:33:04,570 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} is not yet known 09:33:04,570 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, nanosAgo=10610, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:33:04,570 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@7501c41b instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@20891c90 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7568a802 transmitting entry ConnectionEntry{request=CreateLocalHistoryRequest{target=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984]}, enqueuedTicks=790861036210} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$Remote@20891c90 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 for transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=OPEN} allocated sequence 0 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2294/0x00000008016fbb30@367188c8 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7568a802 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY}, enqueuedTicks=790861191793} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY}} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: Message size: 487 bytes, total slices to send: 1 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: Message does not need to be sliced - sending original message 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} allocated sequence 1 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2302/0x000000080170df70@5482e5ec 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7568a802 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE}, enqueuedTicks=790861276204} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: slice: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, message: Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE}} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: Message size: 295 bytes, total slices to send: 1 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard_slicer-id-913: Message does not need to be sliced - sending original message 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.ClientLocalHistory@6ad0a3c5 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@13d1ad7d 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1972850 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@211b7547, splicing queue 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1972850 queue spliced 0 messages 09:33:04,571 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 716.1 μs 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CreateLocalHistoryPayload, isSendImmediate: false 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] LeaderFrontendState - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: created history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: allocating new open transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Opening an unchained snapshot in LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTreeTransactionChain - Allocated read-write transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 snapshot org.opendaylight.yangtools.yang.data.tree.impl.node.Version@3b6a4d92 09:33:04,572 AM [cluster-test-shard-dispatcher-21] [DEBUG] FrontendTransaction - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:04,573 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY}} failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 09:33:04,573 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: request Envelope{sessionId=0, txSequence=2, message=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=0, protocol=SIMPLE}} failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:04,574 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}] 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process}} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY}, enqueuedTicks=790861191793, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process}} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=1, protocol=READY} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: Request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test@127.0.0.1:2559/user/$a#-424782984], modifications=1, protocol=READY} failed to process at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:147) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.yangtools.yang.data.tree.api.SchemaValidationFailedException: Node (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:junk?revision=2014-03-13)junk is not a valid child of (urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars according to the schema. at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.recursivelyVerifyStructure(AbstractNodeContainerModificationStrategy.java:157) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mergeIntoModifiedNode(AbstractNodeContainerModificationStrategy.java:265) at org.opendaylight.yangtools.yang.data.tree.impl.OperationWithModification.merge(OperationWithModification.java:47) at org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTreeModification.merge(InMemoryDataTreeModification.java:101) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.applyModifications(FrontendReadWriteTransaction.java:553) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.ensureReady(FrontendReadWriteTransaction.java:610) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleModifyTransaction(FrontendReadWriteTransaction.java:586) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:191) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ... 33 more 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE}, enqueuedTicks=790861276204, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0}} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-424782984], modifications=0, protocol=SIMPLE} completed with TransactionFailure{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=1, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0} 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 failed org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 Error during phase CAN_COMMIT, starting Abort org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:04,575 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] DOMBrokerTransactionChain - Transaction chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@6209f577 failed. TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION, tag=operation-failed, applicationTag=null, info=null, cause=org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0]]} at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1124) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808) at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) at org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction.lambda$handleReplayedModifyTransactionRequest$16(RemoteProxyTransaction.java:509) at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.access.commands.OutOfOrderRequestException: Expecting request 0 at org.opendaylight.controller.cluster.datastore.FrontendTransaction.replaySequence(FrontendTransaction.java:80) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:114) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) ... 17 more 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ConcurrentDOMDataBroker - Tx: DOM-CHAIN-0-0 aborted successfully 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} directCommit completed 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR}: initiating purge 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-424782984]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@744e8aa4 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, cookie=1} 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=SEALED}: initiating purge 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} allocated sequence 2 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-424782984]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@55a1061a 09:33:04,576 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@7568a802 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-424782984]}, enqueuedTicks=790869524285} 09:33:04,576 AM [main] [DEBUG] DOMBrokerTransactionChain - Chain org.opendaylight.controller.cluster.databroker.DOMBrokerTransactionChain@6209f577 is no longer running 09:33:04,576 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:33:04,576 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,576 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 09:33:04,577 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 09:33:04,577 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,577 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testChainedTransactionFailureWithSingleShard (Follower): Closing 09:33:04,577 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 09:33:04,577 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 09:33:04,577 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:33:04,577 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: purging transaction member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1 09:33:04,577 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 09:33:04,577 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 09:33:04,577 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Replicate message: identifier: member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:04,577 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}] 09:33:04,578 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:33:04,578 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,578 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Issuing gracefulStop to shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientHistory - Force-closing history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=0} java.lang.IllegalStateException: Client ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0} has been shut down at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior.shutdown(AbstractDataStoreClientBehavior.java:113) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:148) at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: onShutDown 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Initiating leader transfer 09:33:04,578 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testChainedTransactionFailureWithSingleShard - waiting on 1 shards 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:33:04,578 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, leaderId=null, leaderPayloadVersion=12] 09:33:04,578 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:04,578 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@659bf5a3 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1795068335] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,578 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-424782984] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-424782984] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Removed transaction chain LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1} 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardDataTree - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@659bf5a3 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Attempting to transfer leadership 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: active: true, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:33:04,578 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,579 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Sending AppendEntries to follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard: AppendEntries [leaderId=member-1-shard-cars-testChainedTransactionFailureWithSingleShard, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=CreateLocalHistoryPayload{identifier=LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testChainedTransactionFailureWithSingleShard, generation=0}, history=1, cookie=1}, size=74}], SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testChainedTransactionFailureWithSingleShard-fe-0-chn-1-txn-0-1, size=75}]]] 09:33:04,579 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.base.messages.LeaderTransitioning] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,579 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,661 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,780 AM [cluster-test-shard-dispatcher-21] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Failed to transfer leadership in 202.1 ms 09:33:04,780 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: In unpauseLeader 09:33:04,780 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: leader transfer failed - sending PoisonPill 09:33:04,780 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader): Leadership transfer expired 09:33:04,781 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Stopping Shard member-1-shard-cars-testChainedTransactionFailureWithSingleShard 09:33:04,781 AM [cluster-test-shard-dispatcher-21] [DEBUG] MessageSlicer - member-1-shard-cars-testChainedTransactionFailureWithSingleShard (Leader)_slicer-id-915: Closing 09:33:04,781 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testChainedTransactionFailureWithSingleShard: Aborting 0 pending queued transactions 09:33:04,781 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] ShardManager - shard-manager-testChainedTransactionFailureWithSingleShard: All shards shutdown - sending PoisonPill to self 09:33:04,781 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testChainedTransactionFailureWithSingleShard 09:33:04,781 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:33:04,781 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: shutting down 09:33:04,781 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] MessageAssembler - member-1-frontend-datastore-testChainedTransactionFailureWithSingleShard: Closing 09:33:04,781 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1828761360] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard#1828761360] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,782 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-1-shard-cars-testChainedTransactionFailureWithSingleShard#867614742] to Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testChainedTransactionFailureWithSingleShard/member-2-shard-cars-testChainedTransactionFailureWithSingleShard] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:04,782 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:04,782 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:04,785 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:04,785 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:04,785 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:04,785 AM [cluster-test-akka.actor.default-dispatcher-24] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:04,788 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:04,802 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:04,802 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:05,402 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:05,802 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [4091770071642645903] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:05,816 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:05,816 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:05,817 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:05.818] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:05,822 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2834138859200947380] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:05,826 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:05,826 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:05,828 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:05.829] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:05,839 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:05,845 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-1377525903554702562] 09:33:05,845 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:05,847 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:05,847 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:05,847 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:05,847 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:05,859 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:05,859 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:05,859 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:05,862 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Slf4jLogger - Slf4jLogger started 09:33:05,867 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-5068572337683604468] 09:33:05,867 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:05,868 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:05,868 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:05,868 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:05,868 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:05,894 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:05,908 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [1833071022440571606] 09:33:05,908 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:05,918 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:05,918 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:05,925 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:05,925 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:05,929 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:05,931 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:05,933 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 09:33:05,938 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:33:05,940 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1111334581] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:33:05,959 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@43c85422 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a89d521 09:33:05,959 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@22d4a882 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1714037585959, None)) 09:33:05,959 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037585958,0,0)) 09:33:05,959 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,959 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@3a89d521 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@550bddeb 09:33:05,960 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 09:33:05,960 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@6d272bf9 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,961 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,961 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521] 09:33:05,962 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 09:33:05,966 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 09:33:05,966 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,966 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@10cbe577 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} 09:33:05,966 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#444193044] 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1544782627]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@f9aeefe 09:33:05,966 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1488500870 created and ready for shard:member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:05,966 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4b060ff4 aborted transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1544782627], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@306e787e 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1} 09:33:05,966 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@10cbe577 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-1-datastore-testForwardedReadyTransactionForwardedToLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1544782627]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@58bb7170 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} has a successor, not scheduling timer 09:33:05,966 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testForwardedReadyTransactionForwardedToLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testForwardedReadyTransactionForwardedToLeader/member-1-shard-cars-testForwardedReadyTransactionForwardedToLeader#-1884521062], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:05,966 AM [cluster-test-akka.actor.default-dispatcher-12] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-1:snapshot-read#444193044] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,967 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithShardLeaderNotResponding 09:33:05,968 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#1488500870] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,970 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:33:05,970 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:05,970 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:05,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 09:33:05,970 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 09:33:05,970 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:33:05,970 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:33:05,970 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 09:33:05,970 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 09:33:05,971 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 09:33:05,971 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,977 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:05,980 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:05,981 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithShardLeaderNotResponding 09:33:05,981 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Creating local shard: member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,982 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:05,984 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:05,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:05,986 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: no initial generation override, starting from 0 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@29b89906 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a892014 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@5f8bbd21 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, 0, 1714037585990, None)) 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037585989,0,0)) 09:33:05,990 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4505a6c2 09:33:05,990 AM [cluster-test-akka.actor.default-dispatcher-6] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1a892014 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@32bcb0a1 09:33:05,994 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} started 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: updated peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding on shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-180166679] 09:33:05,995 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithShardLeaderNotResponding, persistent : true 09:33:05,996 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithShardLeaderNotResponding with journal batch size 1 09:33:05,996 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding#-180166679] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,996 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-246424025] 09:33:05,996 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#43540092] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:05,996 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:05,996 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:05,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Initializing shard [cars] 09:33:05,997 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is initialized 09:33:05,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:33:05,997 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:33:05,997 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,997 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding-notifier#43540092 created and ready for shard:member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:05,997 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from null to Follower 09:33:05,997 AM [cluster-test-akka.actor.default-dispatcher-6] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithShardLeaderNotResponding , registered listener akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding 09:33:05,998 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-2-shard-cars-testTransactionWithShardLeaderNotResponding from null to Follower 09:33:06,1 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding/shard-member-2:snapshot-read#-246424025] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:06,45 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#1111334581] 09:33:06,45 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: findLocalShard : cars 09:33:06,45 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521] 09:33:06,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:06,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Received ElectionTimeout - switching to Candidate 09:33:06,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 09:33:06,206 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,206 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 1 09:33:06,207 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:06,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 09:33:06,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: false, hasLeader : false 09:33:06,207 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding is no longer the leader 09:33:06,207 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Follower to Candidate 09:33:06,207 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Follower to Candidate 09:33:06,457 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:33:06,457 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,457 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 2 09:33:06,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:33:06,776 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,776 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 3 09:33:06,857 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:06,857 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,858 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-3-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,892 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding to akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:06,933 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:07,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Received ElectionTimeout 09:33:07,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,16 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Starting new election term 4 09:33:07,16 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,22 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:07,22 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Cluster state: [] 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=null 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,23 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:07,23 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithShardLeaderNotResponding] 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: onLeadershipChange, isLeader: true, hasLeader : true 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: became leader with frontend state for [] 09:33:07,24 AM [cluster-test-akka.actor.default-dispatcher-14] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithShardLeaderNotResponding , received role change from Candidate to Leader 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=12] 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received role changed for member-1-shard-cars-testTransactionWithShardLeaderNotResponding from Candidate to Leader 09:33:07,24 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 09:33:07,24 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Closing 09:33:07,24 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithShardLeaderNotResponding peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,24 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithShardLeaderNotResponding, leaderId=member-1-shard-cars-testTransactionWithShardLeaderNotResponding, leaderPayloadVersion=12] 09:33:07,24 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:07,24 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:07,25 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All Shards are ready - data store testTransactionWithShardLeaderNotResponding is ready 09:33:07,25 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding Received follower initial sync status for member-2-shard-cars-testTransactionWithShardLeaderNotResponding status sync done false 09:33:07,25 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithShardLeaderNotResponding updated: matchIndex: -1, nextIndex: 0 09:33:07,25 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithShardLeaderNotResponding: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:07,50 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12a6f228: creating a new snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 09:33:07,50 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 09:33:07,50 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:07,50 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:07,50 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] 09:33:07,50 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:07,50 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=0} shard 1 09:33:07,51 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0 09:33:07,51 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:07,51 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=12] 09:33:07,51 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@3b1cf8c7 09:33:07,51 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} scheduling timeout in 2 seconds 09:33:07,51 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding, primaryVersion=12] 09:33:07,51 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6b27b0d2, attempting to connect 09:33:07,52 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} is not yet known 09:33:07,52 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, nanosAgo=10660, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:33:07,53 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2d6969ca instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce91896 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5ce91896 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 09:33:07,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} to successor RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=OPEN} enqueing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1289/0x0000000801352f58@24913961 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1997033529 nanoseconds 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38e77b08 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=793344429831} 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12a6f228 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@2cdef92a 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@12e24d9b superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3f953497, splicing queue 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@12e24d9b queue spliced 0 messages 09:33:07,54 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 650.6 μs 09:33:07,55 AM [cluster-test-shard-dispatcher-28] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: allocating new open snapshot member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1 09:33:07,55 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-917: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 09:33:07,55 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] SlicedMessageState - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-917: Message size: 235 bytes, total slices to send: 1 09:33:07,55 AM [cluster-test-akka.actor.default-dispatcher-14] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding_slicer-id-917: Message does not need to be sliced - sending original message 09:33:07,56 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 09:33:07,56 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=793344429831, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0}} 09:33:07,56 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=0} 09:33:07,56 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:33:07,57 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,57 AM [cluster-test-shard-dispatcher-28] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithShardLeaderNotResponding (Leader)_slicer-id-920: Closing 09:33:07,57 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 09:33:07,57 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:33:07,58 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:07,58 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:07,62 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:07,62 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:07,62 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:07,62 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:07,63 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:07,68 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithShardLeaderNotResponding 09:33:07,68 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12a6f228: creating a new transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 09:33:07,68 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1 for transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 09:33:07,68 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:33:07,68 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:07,68 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@12a6f228 readied transaction member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-0 09:33:07,68 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@760d794a 09:33:07,68 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:07,68 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:07,68 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38e77b08 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=1, protocol=SIMPLE}, enqueuedTicks=793361791441} 09:33:07,68 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=1, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=1, protocol=SIMPLE}} 09:33:07,68 AM [main] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: Message size: 399 bytes, total slices to send: 1 09:33:07,68 AM [main] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: Message does not need to be sliced - sending original message 09:33:08,71 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1377525903554702562] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:08,72 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-1377525903554702562] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:09,61 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 09:33:09,61 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} is empty 09:33:09,61 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out entry TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=1, protocol=SIMPLE}, enqueuedTicks=793361791441, sessionId=0, txSequence=1} 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} timed out 1 tasks 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED} directCommit completed 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=FLUSHED}: initiating purge 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@3ca92756 09:33:09,71 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithShardLeaderNotResponding/member-1-shard-cars-testTransactionWithShardLeaderNotResponding#-1888760521], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 1999939899 nanoseconds 09:33:09,72 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38e77b08 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886]}, enqueuedTicks=795365176059} 09:33:09,72 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:33:09,72 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:09,72 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithShardLeaderNotResponding - waiting on 1 shards 09:33:09,72 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: onShutDown 09:33:09,73 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithShardLeaderNotResponding 09:33:09,73 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithShardLeaderNotResponding (Follower): Closing 09:33:09,73 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithShardLeaderNotResponding: Aborting 0 pending queued transactions 09:33:09,73 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardManager - shard-manager-testTransactionWithShardLeaderNotResponding: All shards shutdown - sending PoisonPill to self 09:33:09,73 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithShardLeaderNotResponding 09:33:09,73 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:33:09,73 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0} 09:33:09,73 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: shutting down 09:33:09,73 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:33:09,74 AM [main] [WARN] ActorUtils - ShardManager for testTransactionWithShardLeaderNotResponding data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:33:09,74 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithShardLeaderNotResponding: Closing 09:33:09,74 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:09,74 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithShardLeaderNotResponding#-380240761] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:09,74 AM [cluster-test-akka.actor.default-dispatcher-26] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1721592886] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1721592886] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:09,74 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:09,74 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:10,93 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-5068572337683604468] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:10,93 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:10,93 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:10,94 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:10.095] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:10,104 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:10,104 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:10,106 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:10.107] [cluster-test-akka.actor.internal-dispatcher-4] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:10,120 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:10,125 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [5626801711123738808] 09:33:10,125 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:10,126 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:10,126 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:10,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:10,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:10,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:10,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:10,140 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:10,143 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:10,150 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [650819541947302380] 09:33:10,150 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:10,152 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:10,152 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:10,160 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:10,160 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:10,166 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:10,171 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-8019844970208748498] 09:33:10,171 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:10,173 AM [cluster-test-akka.actor.default-dispatcher-5] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:10,173 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:10,179 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:10,179 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:10,182 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:10,183 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:10,184 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:10,192 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:10,199 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:10,200 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-669446834] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,202 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:10,207 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:10,208 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:10,215 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 09:33:10,215 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:10,215 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:10,215 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@7b42e21e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2d04a4cd 09:33:10,215 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,215 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@746c936e 09:33:10,215 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037590215, None)) 09:33:10,216 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037590214,0,0)) 09:33:10,216 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:10,216 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,216 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2d04a4cd to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2a351c10 09:33:10,216 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@43364b1b 09:33:10,216 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 09:33:10,216 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:10,218 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:10,218 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:10,218 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,219 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,219 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621] 09:33:10,219 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:33:10,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,219 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:10,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#564359567 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,219 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#58243281] 09:33:10,219 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#564359567] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,220 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:33:10,220 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#58243281] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,220 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:10,220 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:10,221 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:10,221 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:10,221 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:33:10,221 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:33:10,221 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:33:10,221 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:10,221 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:33:10,223 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:10,223 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,224 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: no initial generation override, starting from 0 09:33:10,224 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:10,225 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@33966690 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1926018 09:33:10,225 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@4490fe4e 09:33:10,225 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037590225, None)) 09:33:10,225 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:10,225 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,225 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037590224,0,0)) 09:33:10,225 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@1926018 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@701a4041 09:33:10,225 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:10,225 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,225 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@506dd45f 09:33:10,225 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} started 09:33:10,226 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:10,226 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,226 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,226 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,226 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: updated peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges on shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges whose actor address is Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106] 09:33:10,226 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Shard created : member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:33:10,226 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,226 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#245642313 created and ready for shard:member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,226 AM [cluster-test-akka.actor.default-dispatcher-26] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-1060863259] 09:33:10,226 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#245642313] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,226 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Starting recovery for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:33:10,227 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-2:snapshot-read#-1060863259] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:10,227 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:33:10,227 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:10,227 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,227 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:10,227 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:33:10,276 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-669446834] 09:33:10,276 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:33:10,276 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621] 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 1 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:10,536 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:10,536 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:10,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:10,776 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:10,776 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 2 09:33:10,809 AM [cluster-test-shard-dispatcher-20] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:11,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:11,85 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,85 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 3 09:33:11,147 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:11,147 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,147 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,157 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,195 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:11,325 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:11,325 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,325 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 4 09:33:11,325 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=null 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,327 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:11,328 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:11,328 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:11,328 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [] 09:33:11,329 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 09:33:11,329 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:11,329 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:11,329 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,330 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:11,330 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:11,330 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:11,330 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:11,330 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 09:33:11,330 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: -1, nextIndex: 0 09:33:11,330 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:11,331 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:33:11,332 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 09:33:11,332 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:11,332 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:11,332 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:11,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:11,332 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} shard 1 09:33:11,332 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=12] 09:33:11,332 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:33:11,332 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:11,332 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-0 09:33:11,332 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@60014af5 09:33:11,332 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} scheduling timeout in 2 seconds 09:33:11,332 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, primaryVersion=12] 09:33:11,336 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@306cbf66, attempting to connect 09:33:11,337 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} is not yet known 09:33:11,337 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, nanosAgo=9620, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:11,338 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5c5ddb19 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@679ee132 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@679ee132 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2302/0x000000080170df70@6fe97c0d 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c9d134c transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE}, enqueuedTicks=797626087115} 09:33:11,338 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-923: slice: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE}} 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-923: Message size: 413 bytes, total slices to send: 1 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] MessageSlicer - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges_slicer-id-923: Message does not need to be sliced - sending original message 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1e80c11f 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@62c7534e superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@2ecbf041, splicing queue 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@62c7534e queue spliced 0 messages 09:33:11,339 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 604.0 μs 09:33:11,339 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new open transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: readying transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 ready 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: transitioned member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 to ready 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct canCommit 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 validated 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct preCommit 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 candidate ready 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: canCommit - messages: [] 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 prepared 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 initiating direct commit 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 182 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 submitted to persistence 09:33:11,340 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}]]] 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=182} 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 1 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:11,341 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 0 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=182}] 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@27a74b8c}, purged=MutableUnsignedLongSet{size=0}} 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-125132705] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:11,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-125132705] 09:33:11,342 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 09:33:11,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:33:11,342 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-125132705] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,342 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 84.75 μs 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 0 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE}, enqueuedTicks=797626087115, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] RemoteProxyTransaction - Modification request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=2, protocol=SIMPLE} completed with TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 0 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 0 data Simple{size=182} 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@5428279c 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SEALED}: initiating purge 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 1 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4cb88c3f 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:11,343 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c9d134c transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]}, enqueuedTicks=797636801690} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendMetadata - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@7dcf3c16}, purged=MutableUnsignedLongSet{size=0}} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,343 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1697418910] with name member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 09:33:11,344 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=0 09:33:11,344 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1697418910] 09:33:11,344 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:33:11,344 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 47.25 μs 09:33:11,344 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#1697418910] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 09:33:11,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 09:33:11,344 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:11,345 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testSingleShardTransactionsWithLeaderChanges 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: RaftPolicy used with prev.config org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy, RaftPolicy used with newConfig null 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Re-initializing to Follower with previous leaderId member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:11,345 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : true 09:33:11,345 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:11,346 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 09:33:11,346 AM [cluster-test-shard-dispatcher-25] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,346 AM [cluster-test-shard-dispatcher-25] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-925: Closing 09:33:11,346 AM [cluster-test-shard-dispatcher-25] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:33:11,346 AM [cluster-test-shard-dispatcher-25] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:11,347 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:11,347 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:11,353 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:11,353 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:11,353 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:11,353 AM [cluster-test-akka.actor.default-dispatcher-23] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:11,354 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:11,367 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marked address [akka://cluster-test@127.0.0.1:2558] as [Leaving] 09:33:11,416 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:11,417 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:33:11,417 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106] 09:33:11,417 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,467 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,518 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Leaving - leader is unavailable 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Leaving), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:11,559 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:11,560 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:11,560 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:11,560 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:11,560 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c9d134c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@14bdebc5, splicing queue 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6c9d134c queue spliced 1 messages 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:11,561 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:11,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:11,561 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 09:33:11,573 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:11,580 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-4745313561262380596] 09:33:11,580 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:11,582 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:11,582 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:11,582 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:11,582 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:11,595 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:11,596 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:11,597 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:11,600 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Starting ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:11,605 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#-919175555] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,605 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: recovered identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:11,606 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 09:33:11,606 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@69ee3a3f to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2d9119eb 09:33:11,606 AM [cluster-test-akka.actor.default-dispatcher-19] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6893923d 09:33:11,606 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - Recovery complete : shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:11,606 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Creating local shard: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,606 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, 0, 1714037591606, None)) 09:33:11,607 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037591605,0,0)) 09:33:11,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@13e9fc4c 09:33:11,607 AM [cluster-test-akka.actor.default-dispatcher-18] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@2d9119eb to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5de79331 09:33:11,607 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} started 09:33:11,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:11,607 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,608 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Shard created : member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, persistent : true 09:33:11,608 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - Starting recovery for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with journal batch size 1 09:33:11,608 AM [cluster-test-akka.actor.default-dispatcher-18] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,609 AM [cluster-test-akka.actor.default-dispatcher-15] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-886944104] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=1, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=2, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=3, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=4, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 0, size: 182 09:33:11,610 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.ApplyJournalEntries from Actor[akka://cluster-test/deadLetters] 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received apply journal entries for recovery, applying to state: 0 to 0 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: starting log recovery batch with max size 1 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendMetadata - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@14b49638}, purged=MutableUnsignedLongSet{size=0}} 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:11,611 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-notifier#-886944104 created and ready for shard:member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from -1 to 0 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.SimpleReplicatedLogEntry from Actor[akka://cluster-test/deadLetters] 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Received ReplicatedLogEntry for recovery: index: 1, size: 74 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Recovery completed in 483.0 μs - Switching actor to Follower - last log index = 1, last log term = 4, snapshot index = -1, snapshot term = -1, journal size = 2 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:11,611 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:11,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Initializing shard [cars] 09:33:11,611 AM [cluster-test-shard-dispatcher-11] [DEBUG] ShardInformation - Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is initialized 09:33:11,611 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from null to Follower 09:33:11,611 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , registered listener akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges 09:33:11,612 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1505799487] 09:33:11,612 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/shard-member-1:snapshot-read#1505799487] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:11,612 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from null to Follower 09:33:11,657 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:11,658 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: findLocalShard : cars 09:33:11,658 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106] 09:33:11,729 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:11,729 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,729 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 6 09:33:11,729 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,779 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 09:33:11,779 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 09:33:11,780 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:11,780 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:11,780 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:11,781 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:11,859 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:11,859 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:11,859 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 7 09:33:11,859 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,60 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,60 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,60 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 8 09:33:12,60 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=5, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 5 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 5 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:12,142 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:12,142 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:12,189 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,189 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,189 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 9 09:33:12,189 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,359 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,359 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,359 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 10 09:33:12,359 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,359 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [5626801711123738808] but current is [-4745313561262380596]. Reason: ActorSystem terminated 09:33:12,365 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [5626801711123738808] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:12,385 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.GossipEnvelope] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [650819541947302380]. This system with UID [-4745313561262380596] was probably restarted. Messages will be accepted when new handshake has been completed.. [5] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 09:33:12,386 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Replicator - Ignoring message [Replicator$Internal$Status] from [Actor[akka://cluster-test@127.0.0.1:2559/system/clusterReceptionist/replicator#-581454374]] intended for system uid [5626801711123738808], self uid is [-4745313561262380596] 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=5, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 5, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=5, voteGranted=false] 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 5 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 6 in "RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 5 - switching to Follower 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=6, votedFor=null 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=6, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 6, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=6, voteGranted=false] 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 6 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:12,386 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:12,386 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 7 in "RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 6 - updating term 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=7, votedFor=null 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=7, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 7, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,387 AM [cluster-test-shard-dispatcher-11] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=7, voteGranted=false] 09:33:12,387 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=5, voteGranted=false], my term is 10 09:33:12,387 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=5, voteGranted=false], current voteCount: 1 09:33:12,387 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=6, voteGranted=false], my term is 10 09:33:12,387 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=6, voteGranted=false], current voteCount: 1 09:33:12,387 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Leaving)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:33:12,387 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Marking unreachable node [akka://cluster-test@127.0.0.1:2558] as [Down] 09:33:12,388 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Received UnreachableMember: memberName MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 8 in "RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 7 - updating term 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=8, votedFor=null 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=8, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 8, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=8, voteGranted=false] 09:33:12,389 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=7, voteGranted=false], my term is 10 09:33:12,389 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=7, voteGranted=false], current voteCount: 1 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 9 in "RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 8 - updating term 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=9, votedFor=null 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=9, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 9, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,389 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=9, voteGranted=false] 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=8, voteGranted=false], my term is 10 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=8, voteGranted=false], current voteCount: 1 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 10 in "RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 9 - updating term 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=10, votedFor=null 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=10, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 10, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,390 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=10, voteGranted=false] 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=9, voteGranted=false], my term is 10 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=9, voteGranted=false], current voteCount: 1 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=10, voteGranted=false], my term is 10 09:33:12,390 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=10, voteGranted=false], current voteCount: 1 09:33:12,428 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] LocalActorRefProvider$$anon$2 - Message [akka.cluster.ddata.Replicator$Internal$Status] wrapped in [akka.actor.ActorSelectionMessage] to Actor[akka://cluster-test/] was dropped. Unknown system with UID [-8019844970208748498]. This system with UID [-4745313561262380596] was probably restarted. Messages will be accepted when new handshake has been completed.. [6] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:12,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,539 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 11 09:33:12,539 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 11 in "RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 10 - updating term 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=11, votedFor=null 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=11, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 11, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,540 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=11, voteGranted=false] 09:33:12,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=11, voteGranted=false], my term is 11 09:33:12,541 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=11, voteGranted=false], current voteCount: 1 09:33:12,602 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - New incarnation of existing member [Member(akka://cluster-test@127.0.0.1:2558, Down)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join. 09:33:12,679 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,679 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,679 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 12 09:33:12,680 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 12 in "RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 11 - updating term 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=12, votedFor=null 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=12, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 12, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,681 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=12, voteGranted=false] 09:33:12,682 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=12, voteGranted=false], my term is 12 09:33:12,682 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=12, voteGranted=false], current voteCount: 1 09:33:12,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:12,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:12,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:12,800 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:12,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:12,800 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Candidate, leaderId=null, activeMember=true] 09:33:12,869 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:12,869 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,869 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 13 09:33:12,869 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 13 in "RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 12 - updating term 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=13, votedFor=null 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=13, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 13, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:12,871 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=13, voteGranted=false] 09:33:12,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=13, voteGranted=false], my term is 13 09:33:12,872 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=13, voteGranted=false], current voteCount: 1 09:33:13,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [] self member: Member(akka://cluster-test@127.0.0.1:2558, Removed) 09:33:13,1 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 14 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:13,2 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:13,2 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=14, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 14 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 09:33:13,20 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 09:33:13,20 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:13,20 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:13,21 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:13,21 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:13,21 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 09:33:13,21 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=14, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 14, votedFor: member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastIndex: 1, lastTerm: 4 09:33:13,21 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=14, voteGranted=false] 09:33:13,22 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=14, voteGranted=false], my term is 14 09:33:13,22 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=14, voteGranted=false], current voteCount: 1 09:33:13,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:13,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,230 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 15 09:33:13,230 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4], my term is 14 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 15 in "RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than Candidate's term 14 - switching to Follower 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=15, votedFor=null 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=15, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 15, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=15, voteGranted=false] 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 15 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:13,231 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:13,232 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 09:33:13,232 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 09:33:13,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=15, voteGranted=false], my term is 15 09:33:13,232 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=15, voteGranted=false], current voteCount: 1 09:33:13,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is removing unreachable node [akka://cluster-test@127.0.0.1:2558] 09:33:13,236 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [5626801711123738808] but current is [-4745313561262380596]. Reason: Cluster member removed, previous status [Down] 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} is empty 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1} 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} is empty 09:33:13,349 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: not scheduling timeout on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:13,420 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:13,420 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,420 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 16 09:33:13,420 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 16 in "RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 15 - updating term 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=16, votedFor=null 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=16, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 16, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:13,421 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=16, voteGranted=false] 09:33:13,422 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=16, voteGranted=false], my term is 16 09:33:13,423 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=16, voteGranted=false], current voteCount: 1 09:33:13,574 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberRemoved: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:13,574 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Association - Quarantine of [akka://cluster-test@127.0.0.1:2558] ignored due to non-matching UID, quarantine requested for [5626801711123738808] but current is [-4745313561262380596]. Reason: Cluster member removed, previous status [Down] 09:33:13,579 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: running timer on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:13,579 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 2 seconds 09:33:13,609 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:13,609 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,609 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 17 09:33:13,609 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:13,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:13,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] with higher term appears reachable, updating term. 09:33:13,610 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 17 in "RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4]" message is greater than follower's term 16 - updating term 09:33:13,610 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=17, votedFor=null 09:33:13,611 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=17, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=0, lastLogTerm=4] - currentTerm: 17, votedFor: null, lastIndex: 1, lastTerm: 4 09:33:13,611 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=17, voteGranted=false] 09:33:13,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=17, voteGranted=false], my term is 17 09:33:13,611 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=17, voteGranted=false], current voteCount: 1 09:33:13,623 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING, roles [member-1, dc-default], version [0.0.0] 09:33:13,625 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Welcome from [akka://cluster-test@127.0.0.1:2559] 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-3-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,627 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,801 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2558, Joining) 09:33:13,801 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:13,801 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:13,801 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,801 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 18 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=18, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] to peer member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 18 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:13,802 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:13,802 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVote [term=18, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4], my term is 17 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Term 18 in "RequestVote [term=18, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4]" message is greater than Candidate's term 17 - switching to Follower 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=null 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): In requestVote: RequestVote [term=18, candidateId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=1, lastLogTerm=4] - currentTerm: 18, votedFor: null, lastIndex: 0, lastTerm: 4 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=18, votedFor=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): requestVote returning: RequestVoteReply [term=18, voteGranted=true] 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Follower, election term: 18 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:13,803 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Follower 09:33:13,803 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Follower 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=18, voteGranted=true], my term is 18 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=18, voteGranted=true], current voteCount: 1 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): LastApplied index 0 is behind last index 1 - switching to PreLeader 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to PreLeader, election term: 18 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:33:13,804 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Replicate message: identifier: null, logIndex: 2, payload: class org.opendaylight.controller.cluster.raft.persisted.NoopPayload, isSendImmediate: true 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 0, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:33:13,805 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to PreLeader 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to PreLeader 09:33:13,805 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:33:13,805 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:13,805 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,806 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:13,806 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:13,806 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 09:33:13,806 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 09:33:13,806 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:13,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 0, nextIndex: 1 09:33:13,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:33:13,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Sending AppendEntries to follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:33:13,806 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}], SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b]]] 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 2 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 3 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=18, success=true, followerId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=18, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}] 09:33:13,808 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=2, term=18, payload=org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b] 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Not updating commit index to 1 - retrieved log entry with index 1, term 4 does not match the current term 18 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 1 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: replication of member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 has no callback 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Setting last applied to 2 09:33:13,808 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 18 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader): Received ApplyState [identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, replicatedLogEntry=SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74}]] - lastApplied: 2, lastIndex: 2 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader) :- Switching from behavior PreLeader to Leader, election term: 18 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (PreLeader)_slicer-id-928: Closing 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 09:33:13,809 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from PreLeader to Leader 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from PreLeader to Leader 09:33:13,809 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 2 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 2 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 1 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 0 to 1 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, size=74} 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Purged transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 2 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 1 to 2 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 2 data org.opendaylight.controller.cluster.raft.persisted.NoopPayload@116bda6b 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 1 for term 18 09:33:13,810 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=2 09:33:13,823 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:13,823 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8: creating a new transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:33:13,823 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:33:13,823 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:33:13,823 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8 readied transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-0 09:33:13,823 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=1, protocol=SIMPLE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6d4e7e43 09:33:13,823 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 09:33:14,39 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:14,39 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:14,39 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:14,39 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:14,39 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:14,40 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Follower, leaderId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, activeMember=true] 09:33:14,255 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:14,260 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received ShardNotInitializedTimeout message for shard cars 09:33:14,260 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Returning NoShardLeaderException for shard cars 09:33:14,260 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:14,260 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:14,260 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:14,260 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:14,647 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:14,647 AM [cluster-test-shard-dispatcher-12] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:14,647 AM [cluster-test-shard-dispatcher-12] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:14,647 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking for leader akka://cluster-test@127.0.0.1:2558 in the cluster unreachable set [] 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Leader akka://cluster-test@127.0.0.1:2558 cluster status is Joining - leader is unavailable 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Up), Member(akka://cluster-test@127.0.0.1:2558, Joining), Member(akka://cluster-test@127.0.0.1:2559, Up)] self member: Member(akka://cluster-test@127.0.0.1:2559, Up) 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Received ElectionTimeout - switching to Candidate 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Election: Candidate has following voting peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 19 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower) :- Switching from behavior Follower to Candidate, election term: 19 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:14,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:14,880 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Follower to Candidate 09:33:14,880 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 09:33:14,880 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:14,880 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Follower to Candidate 09:33:14,880 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:14,880 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:14,880 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Cluster state: [] 09:33:14,880 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Candidate in requestVote:RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18] with higher term appears reachable, updating term. 09:33:14,880 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Term 19 in "RequestVote [term=19, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18]" message is greater than leader's term 18 - switching to Follower 09:33:14,880 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=19, votedFor=null 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) :- Switching from behavior Leader to Follower, election term: 19 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-929: Closing 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: false, hasLeader : false 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges is no longer the leader 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: removing frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:14,881 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Leader to Follower 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=-1] 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:14,881 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges from Leader to Follower 09:33:15,50 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Received ElectionTimeout 09:33:15,50 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,50 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Starting new election term 20 09:33:15,50 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): Sending RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18] to peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Cluster state: [] 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Candidate in requestVote:RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18] with higher term appears reachable, updating term. 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Term 20 in "RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18]" message is greater than follower's term 19 - updating term 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=null 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): In requestVote: RequestVote [term=20, candidateId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, lastLogIndex=2, lastLogTerm=18] - currentTerm: 20, votedFor: null, lastIndex: 2, lastTerm: 18 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Set currentTerm=20, votedFor=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,51 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): requestVote returning: RequestVoteReply [term=20, voteGranted=true] 09:33:15,52 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): RaftRPC message received RequestVoteReply [term=20, voteGranted=true], my term is 20 09:33:15,52 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate): handleRequestVoteReply: RequestVoteReply [term=20, voteGranted=true], current voteCount: 1 09:33:15,52 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Election: Leader has following peers: [member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges] 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 2, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Candidate) :- Switching from behavior Candidate to Leader, election term: 20 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onLeadershipChange, isLeader: true, hasLeader : true 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: became leader with frontend state for [member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges] 09:33:15,53 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges , received role change from Candidate to Leader 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received role changed for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges from Candidate to Leader 09:33:15,53 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:15,53 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: getLeaderAddress leaderId = member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges peerAddress = akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderPayloadVersion=12] 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:15,54 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done false 09:33:15,54 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 2, nextIndex: 3 09:33:15,55 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 3, leaderLastIndex: 2, leaderSnapShotIndex: 1 09:33:15,55 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges Received follower initial sync status for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges status sync done true 09:33:15,171 AM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=20, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=18, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 115, lastApplied : 2, commitIndex : 2 09:33:15,253 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:15,253 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,253 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges to akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,253 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All Shards are ready - data store testSingleShardTransactionsWithLeaderChanges is ready 09:33:15,253 AM [cluster-test-shard-dispatcher-20] [INFO] TestShard - Peer address for peer member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,279 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:15,279 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:15,279 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:15,279 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:15,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:15,279 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7933eb27] 09:33:15,279 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2559/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@7933eb27] 09:33:15,280 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@7836ca88, attempting to connect 09:33:15,280 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@679ee132 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@490e6368 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} previous state DONE 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@490e6368 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=OPEN}: initiating purge 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} allocated sequence 0 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@371bf1a0 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d34c108 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756]}, enqueuedTicks=797636801690} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@490e6368 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=1, protocol=SIMPLE} to successor LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=OPEN} 09:33:15,280 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], modifications=1, protocol=SIMPLE} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} allocated sequence 0 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@6d4e7e43 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d34c108 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], coordinated=false}, enqueuedTicks=800117121316} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8 finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@1274391 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@72acb0f1 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@1ab84e2a, splicing queue 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@72acb0f1 queue spliced 0 messages 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 903.3 μs 09:33:15,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1 already purged 09:33:15,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] AbstractFrontendHistory - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: allocating new ready transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756]}, enqueuedTicks=797636801690, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionPurgeResponse{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, sequence=0}} 09:33:15,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct canCommit 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE}: purge completed 09:33:15,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Validating transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@490e6368 purged transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=DONE} 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@679ee132 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 09:33:15,281 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 validated 09:33:15,281 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:15,282 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5c5ddb19 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-0-1, state=SUCCESSOR} 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct preCommit 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Preparing transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 candidate ready 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - candidate rootPath: / 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: canCommit - messages: [] 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: preCommit - successfulFromPrevious: [] 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 prepared 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendReadWriteTransaction - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 initiating direct commit 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Starting commit for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 241 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}] 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}] 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: false, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:33:15,282 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 submitted to persistence 09:33:15,285 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}] 09:33:15,286 AM [cluster-test-shard-dispatcher-20] [WARN] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=20, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=2, logLastTerm=18, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 114, lastApplied : 2, commitIndex : 2 09:33:15,286 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 3 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,286 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=18, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}]]] 09:33:15,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=2, prevLogTerm=18, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}]]] 09:33:15,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Number of entries to be appended = 1 09:33:15,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:33:15,286 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Append entry to log Simple{size=241} 09:33:15,287 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}] 09:33:15,287 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Log size is now 2 09:33:15,287 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): handleAppendEntries returning : AppendEntriesReply [term=20, success=true, followerId=member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges, logLastIndex=3, logLastTerm=20, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:15,287 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=3, term=20, payload=Simple{size=241}] 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): handleAppendEntriesReply - FollowerLogInformation for member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges updated: matchIndex: 3, nextIndex: 4 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Setting last applied to 3 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data Simple{size=241} 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Resuming commit of transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1: commit - successfulFromPrevious: [] 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 changing state from COMMIT_PENDING to COMMITTED 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756], coordinated=false}, enqueuedTicks=800117121316, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionCommitSuccess{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=0}} 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 2 for term 20 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@64e91691 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} has a successor, not scheduling timer 09:33:15,287 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 4, leaderLastIndex: 3, leaderSnapShotIndex: 2 09:33:15,287 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#288529621], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:15,287 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@469e30b8: creating a new snapshot member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 09:33:15,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=3 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=SEALED}: initiating purge 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@5f69ea56 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: publishChanges: rootPath: / 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d34c108 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#1281953756]}, enqueuedTicks=801581202526} 09:33:15,288 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Elapsed time for generation of change events for member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher: 50.46 μs 09:33:15,288 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1 for transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0 09:33:15,288 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=1} 09:33:15,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: purging transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persist data SimpleReplicatedLogEntry [index=4, term=20, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 09:33:15,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=20, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 09:33:15,288 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Replicate message: identifier: member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Commit index set to 3 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 3, context.getLastApplied(): 2, lastIndex(): 3 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Setting last applied to 3 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Moving last applied index from 2 to 3 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying state for log index 3 data Simple{size=241} 09:33:15,288 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Applying foreign transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, history=0, cookie=1} 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Committed transaction member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1573273586] with name member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher 09:33:15,289 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1573273586] 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: fakeSnapshot purging log to 2 for term 20 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Persisting ApplyJournalEntries with index=3 09:33:15,289 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 4 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,289 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Sending AppendEntries to follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: AppendEntries [leaderId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, prevLogIndex=3, prevLogTerm=20, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=20, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}]]] 09:33:15,289 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1573273586] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,289 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] LocalActorRef - Message [org.opendaylight.controller.cluster.datastore.ShardDataTreeNotificationPublisherActor$PublishNotifications] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1573273586] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges-DTCL-publisher#-1573273586] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,289 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: persist complete SimpleReplicatedLogEntry [index=4, term=20, payload=PurgeTransactionPayload{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-1-1, size=75}] 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges (Follower): Closing 09:33:15,289 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:33:15,289 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 09:33:15,289 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessagesSuccessful$] from Actor[akka://cluster-test/system/in-memory-journal#-1209470493] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,290 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] LocalActorRef - Message [akka.persistence.JournalProtocol$WriteMessageSuccess] from Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges#-1054977728] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,290 AM [cluster-test-shard-dispatcher-12] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:15,290 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,290 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:15,290 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 09:33:15,290 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Issuing gracefulStop to shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Shutting down ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges - waiting on 1 shards 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: onShutDown 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Initiating leader transfer 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@176656ab 09:33:15,290 AM [cluster-test-shard-dispatcher-20] [DEBUG] ShardDataTree - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@176656ab 09:33:15,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Attempting to transfer leadership 09:33:15,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Checking sendAppendEntries for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 09:33:15,291 AM [cluster-test-shard-dispatcher-20] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): sendAppendEntries: 4 is present for follower member-1-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,295 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderId=null, leaderPayloadVersion=12] 09:33:15,295 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:15,295 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:15,295 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d34c108 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@4855e272, splicing queue 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 2 seconds 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@d34c108 queue spliced 1 messages 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] ClientActorBehavior - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: refreshing backend for shard 1 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:15,296 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] 09:33:15,296 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:15,296 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: Scheduling 200 ms timer to wait for shard ShardInformation [shardId=member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges, leaderAvailable=true, actorInitialized=true, followerSyncStatus=true, role=Leader, leaderId=null, activeMember=true] 09:33:15,409 AM [cluster-test-shard-dispatcher-22] [WARN] RaftActorLeadershipTransferCohort - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Failed to transfer leadership in 118.9 ms 09:33:15,409 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: In unpauseLeader 09:33:15,409 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: leader transfer failed - sending PoisonPill 09:33:15,409 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader): Leadership transfer expired 09:33:15,410 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges 09:33:15,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges (Leader)_slicer-id-930: Closing 09:33:15,410 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges: Aborting 0 pending queued transactions 09:33:15,410 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testSingleShardTransactionsWithLeaderChanges: All shards shutdown - sending PoisonPill to self 09:33:15,410 AM [cluster-test-shard-dispatcher-20] [INFO] ShardManager - Stopping ShardManager shard-manager-testSingleShardTransactionsWithLeaderChanges 09:33:15,410 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,410 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0} 09:33:15,411 AM [main] [WARN] ActorUtils - ShardManager for testSingleShardTransactionsWithLeaderChanges data store did not shutdown gracefully java.lang.IllegalStateException: cannot enqueue after timer shutdown at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:154) at akka.actor.Scheduler.scheduleOnce(Scheduler.scala:479) at akka.actor.Scheduler.scheduleOnce$(Scheduler.scala:476) at akka.actor.LightArrayRevolverScheduler.scheduleOnce(LightArrayRevolverScheduler.scala:40) at akka.pattern.PromiseActorRef$.apply(AskSupport.scala:722) at akka.pattern.GracefulStopSupport.gracefulStop(GracefulStopSupport.scala:52) at akka.pattern.GracefulStopSupport.gracefulStop$(GracefulStopSupport.scala:49) at akka.pattern.package$.gracefulStop(package.scala:42) at akka.pattern.Patterns$.gracefulStop(Patterns.scala:404) at akka.pattern.Patterns.gracefulStop(Patterns.scala) at org.opendaylight.controller.cluster.datastore.utils.ActorUtils.shutdown(ActorUtils.java:424) at org.opendaylight.controller.cluster.datastore.AbstractDataStore.close(AbstractDataStore.java:242) at org.opendaylight.controller.cluster.datastore.DistributedDataStoreRemotingIntegrationTest.tearDown(DistributedDataStoreRemotingIntegrationTest.java:207) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at org.junit.runner.JUnitCore.run(JUnitCore.java:115) at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42) at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80) at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141) at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103) at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:56) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:184) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:148) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:122) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) Caused by: akka.actor.SchedulerException: cannot enqueue after timer shutdown 09:33:15,411 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: shutting down 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: Closing 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges#154756454] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#1281953756] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#1281953756] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:15,411 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:15,412 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:15,415 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:15,415 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:15,415 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:15,415 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:15,417 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:15,434 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:15,434 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:16,425 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-4745313561262380596] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:16,446 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [650819541947302380] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:16,451 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:16,451 AM [cluster-test-akka.actor.default-dispatcher-29] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:16,453 AM [cluster-test-akka.actor.default-dispatcher-29] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:16.454] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:16,460 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:16,460 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:16,462 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:16.463] [cluster-test-akka.actor.internal-dispatcher-2] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:16,480 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:16,486 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-8085880534945855403] 09:33:16,486 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:16,488 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:16,488 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:16,495 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:16,495 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:16,499 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:16,499 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:16,499 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:16,502 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:16,506 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [-2163535670106646647] 09:33:16,506 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:16,508 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:16,508 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:16,513 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:16,513 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:16,521 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:16,525 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [-2403884906090605383] 09:33:16,525 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:16,534 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:16,535 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:16,539 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:16,539 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:16,539 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:16,539 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:16,540 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:33:16,542 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:16,544 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,545 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:16,546 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 09:33:16,546 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:16,546 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@17cd9e5e to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@340b479d 09:33:16,546 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@6bdf62e9 09:33:16,546 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:33:16,546 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,546 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:16,546 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,546 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037596546, None)) 09:33:16,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037596545,0,0)) 09:33:16,547 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@4ef91970 09:33:16,547 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@340b479d to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2a197368 09:33:16,547 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 09:33:16,547 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:16,548 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:16,550 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:33:16,550 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:16,550 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,550 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,550 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,551 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-1-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:16,551 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-1-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:33:16,551 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,551 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-1-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:33:16,551 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#517353662 created and ready for shard:member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,551 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-notifier#517353662] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,551 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1197103988] 09:33:16,552 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/shard-member-1:snapshot-read#1197103988] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:33:16,552 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:33:16,552 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testTransactionWithIsolatedLeader is initialized 09:33:16,552 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:33:16,552 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:33:16,554 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1768809694] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,555 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:16,559 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: no initial generation override, starting from 0 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@10416b6b to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5af4d15c 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@41c971b7 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037596560, None)) 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037596559,0,0)) 09:33:16,560 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:33:16,560 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:16,560 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5af4d15c to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@2df75f28 09:33:16,560 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@326547a9 09:33:16,561 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} started 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-127211475] 09:33:16,561 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-127211475] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#1979774682] 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#242098615 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#242098615] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,562 AM [cluster-test-shard-dispatcher-21] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:33:16,562 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:16,562 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:16,562 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:33:16,562 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 09:33:16,562 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#1979774682] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:16,562 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:16,562 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:33:16,562 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,563 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:33:16,563 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:33:16,563 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:33:16,563 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:16,611 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:16,611 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 09:33:16,611 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Received ElectionTimeout - switching to Candidate 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=1, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 1 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageAssembler - member-1-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:33:16,849 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Follower to Candidate 09:33:16,849 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Follower to Candidate 09:33:17,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:33:17,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=2, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,119 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 2 09:33:17,130 AM [cluster-test-shard-dispatcher-21] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:17,408 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:33:17,408 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=3, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,408 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 3 09:33:17,509 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:17,509 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,510 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Peer address for peer member-2-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,514 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,551 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:17,659 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Received ElectionTimeout 09:33:17,659 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,659 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Starting new election term 4 09:33:17,659 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Cluster state: [] 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testTransactionWithIsolatedLeader, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,660 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:17,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:17,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:17,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:33:17,661 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: became leader with frontend state for [] 09:33:17,662 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Candidate to Leader 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Candidate to Leader 09:33:17,662 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:33:17,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 09:33:17,662 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:33:17,662 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:17,662 AM [cluster-test-shard-dispatcher-21] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:17,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:33:17,662 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 09:33:17,663 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: -1, nextIndex: 0 09:33:17,663 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:17,667 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:33:17,667 AM [main] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} 09:33:17,667 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:17,667 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:17,668 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:17,668 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:17,668 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} shard 1 09:33:17,668 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fa4c939] 09:33:17,668 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:33:17,668 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:17,668 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 09:33:17,668 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 09:33:17,668 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:33:17,668 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c: creating a new transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:33:17,668 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:33:17,668 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} allocated sequence 0 09:33:17,672 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fa4c939] 09:33:17,673 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@20aef21b, attempting to connect 09:33:17,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} is not yet known 09:33:17,673 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: created state Enabled{clientId=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, nanosAgo=9280, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:17,673 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4149752c instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state OPEN 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state OPEN 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state OPEN 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 09:33:17,673 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6a481d18 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2b3a3e50 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3bf69fec, splicing queue 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@2b3a3e50 queue spliced 0 messages 09:33:17,674 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 534.3 μs 09:33:17,678 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:17,679 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,679 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:33:17,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:33:17,679 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:17,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:33:17,679 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:33:17,679 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:33:17,679 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:17,680 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:17,680 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1768809694] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1768809694] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,680 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], modifications=1} 09:33:17,680 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:33:17,680 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:17,680 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-1166847725] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-1166847725] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,680 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1768809694] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#1768809694] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,680 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-0 09:33:17,681 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 09:33:17,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:17,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:17,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: findLocalShard : cars 09:33:17,681 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:33:17,681 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,681 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,732 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,780 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,783 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,833 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,884 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: At least 1 followers need to be active, Switching member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: false, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader) :- Switching from behavior Leader to IsolatedLeader, election term: 4 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-935: Closing 09:33:17,899 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:33:17,899 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from Leader to IsolatedLeader 09:33:17,900 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from Leader to IsolatedLeader 09:33:17,900 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,900 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.raft.messages.AppendEntries] from Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:17,934 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.raft.client.messages.GetOnDemandRaftState to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] 09:33:17,935 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], modifications=1} 09:33:17,935 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-0 09:33:17,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} allocated sequence 0 09:33:17,935 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@41d9f73b 09:33:17,935 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:17,935 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2686583c transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=804229094381} 09:33:17,936 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: not currently active leader, rejecting request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}}. isLeader: true, isLeaderActive: false,isLeadershipTransferInProgress: false, paused: false 09:33:17,936 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: request Envelope{sessionId=0, txSequence=0, message=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}} failed org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} indicated no leadership, reconnecting it org.opendaylight.controller.cluster.access.commands.NotLeaderException: Actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785] is not the current leader at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:631) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2686583c superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@60863edd, splicing queue 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@2686583c queue spliced 1 messages 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:17,936 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 09:33:17,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:17,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:17,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:17,937 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:17,937 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:17,937 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:18,348 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:18,348 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:18,348 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:18,349 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:18,349 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:18,349 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:18,529 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is no longer leader 09:33:19,369 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:19,369 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:19,369 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:19,369 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:19,369 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:19,369 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:19,789 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:19,789 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:19,789 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:19,790 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:19,790 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:19,790 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:20,808 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:20,808 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:20,808 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:20,808 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:20,809 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:20,809 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:21,229 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:21,229 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:21,229 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:21,229 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:21,230 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:21,230 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:22,249 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:22,249 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:22,249 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:22,249 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:22,249 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:22,249 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:22,668 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:22,668 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:22,668 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:22,669 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:22,669 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:22,669 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:23,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:23,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:23,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:23,689 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:23,689 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:23,689 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:24,108 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:24,108 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:24,108 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:24,109 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:24,109 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:24,109 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:25,129 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:25,129 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:25,129 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:25,129 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:25,129 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:25,129 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:25,549 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:25,549 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:25,549 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:25,550 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:25,550 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:25,550 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:26,569 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:26,569 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:26,569 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:26,569 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:26,569 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:26,569 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:26,989 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:26,989 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:26,989 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:26,990 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:26,990 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:26,990 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:28,9 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:28,9 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:28,9 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:28,9 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:28,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:28,9 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:28,429 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:28,429 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:28,429 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:28,430 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:28,430 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:28,430 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:29,449 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:29,449 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:29,449 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:29,449 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:29,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:29,449 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 400 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=IsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, activeMember=true] 09:33:29,868 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received ShardNotInitializedTimeout message for shard cars 09:33:29,868 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Returning NoShardLeaderException for shard cars 09:33:29,869 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars failed to resolve org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) 09:33:29,869 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] ModuleShardBackendResolver - Resolution of cookie 1 shard cars failed, removing state java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:29,869 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timed out resolving shard 1, scheduling retry in 1 second java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:29,869 AM [ForkJoinPool.commonPool-worker-4] [WARN] AbstractShardBackendResolver - Failed to resolve shard java.util.concurrent.TimeoutException: Shard has no current leader at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.wrap(AbstractShardBackendResolver.java:151) at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractShardBackendResolver.lambda$resolveBackendInfo$3(AbstractShardBackendResolver.java:134) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: org.opendaylight.controller.cluster.datastore.exceptions.NoShardLeaderException: Shard member-1-shard-cars-testTransactionWithIsolatedLeader currently has no leader. Try again later. at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.onShardNotInitializedTimeout(ShardManager.java:653) at org.opendaylight.controller.cluster.datastore.shardmanager.ShardManager.handleCommand(ShardManager.java:234) at org.opendaylight.controller.cluster.datastore.shardmanager.TestShardManager.handleCommand(TestShardManager.java:28) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ... 5 more 09:33:29,936 AM [main] [INFO] AbstractDataStore - DatastoreContext updated for data store testTransactionWithIsolatedLeader 09:33:29,936 AM [main] [DEBUG] LocalReadWriteProxyTransaction - Applying forwarded request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], modifications=1} 09:33:29,937 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c readied transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-0 09:33:29,937 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:29,938 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardInformation - Sending new DatastoreContext to member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:29,938 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: RaftPolicy used with prev.config null, RaftPolicy used with newConfig null 09:33:29,940 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:29,943 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testTransactionWithIsolatedLeader 09:33:29,944 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: recovered identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:29,945 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 09:33:29,945 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Starting ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:29,945 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2f529110 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c5a9ce7 09:33:29,945 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@48530cb 09:33:29,945 AM [cluster-test-akka.actor.default-dispatcher-22] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testTransactionWithIsolatedLeader, 0, 1714037609945, None)) 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Recovery complete : shard-manager-testTransactionWithIsolatedLeader 09:33:29,946 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testTransactionWithIsolatedLeader: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037609944,0,0)) 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Creating local shard: member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,946 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@4c5a9ce7 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@5c000e1a 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:29,946 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} started 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,946 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@1fbbaefb 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} allocated sequence 0 09:33:29,947 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@3e6eaeb3 09:33:29,947 AM [main] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-1-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-2133844052] 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-2-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-2} with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-2133844052] 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testTransactionWithIsolatedLeader with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader to akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: updated peer member-3-shard-cars-testTransactionWithIsolatedLeader on member MemberName{name=member-3} with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testTransactionWithIsolatedLeader/member-3-shard-cars-testTransactionWithIsolatedLeader on shard member-2-shard-cars-testTransactionWithIsolatedLeader whose actor address is Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader#-2133844052] 09:33:29,947 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Shard created : member-2-shard-cars-testTransactionWithIsolatedLeader, persistent : true 09:33:29,948 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Starting recovery for member-2-shard-cars-testTransactionWithIsolatedLeader with journal batch size 1 09:33:29,948 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/shard-member-2:snapshot-read#1758290098] 09:33:29,948 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-notifier#-693481913 created and ready for shard:member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=null 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class org.opendaylight.controller.cluster.raft.persisted.UpdateElectionTerm from Actor[akka://cluster-test/deadLetters] 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Set currentTerm=4, votedFor=member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:29,949 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Initializing shard [cars] 09:33:29,949 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardInformation - Shard member-2-shard-cars-testTransactionWithIsolatedLeader is initialized 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: false, hasLeader : false 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testTransactionWithIsolatedLeader is no longer the leader 09:33:29,949 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , received role change from null to Follower 09:33:29,949 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Peer address for peer member-1-shard-cars-testTransactionWithIsolatedLeader set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:29,949 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testTransactionWithIsolatedLeader , registered listener akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader 09:33:29,950 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-2-shard-cars-testTransactionWithIsolatedLeader from null to Follower 09:33:30,40 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader: Closing 09:33:30,40 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: getLeaderAddress leaderId = member-1-shard-cars-testTransactionWithIsolatedLeader peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:30,41 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testTransactionWithIsolatedLeader, leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderPayloadVersion=12] 09:33:30,41 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:30,41 AM [cluster-test-shard-dispatcher-31] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:30,41 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:33:30,41 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done false 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [WARN] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) : handleAppendEntriesReply delayed beyond election timeout, appendEntriesReply : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=-1, logLastTerm=-1, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5], timeSinceLastActivity : 12378, lastApplied : -1, commitIndex : -1 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - IsolatedLeader member-1-shard-cars-testTransactionWithIsolatedLeader switching from IsolatedLeader to Leader 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Election: Leader has following peers: [member-2-shard-cars-testTransactionWithIsolatedLeader] 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader) :- Switching from behavior IsolatedLeader to Leader, election term: 4 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (IsolatedLeader)_slicer-id-936: Closing 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testTransactionWithIsolatedLeader: onLeadershipChange, isLeader: true, hasLeader : true 09:33:30,42 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testTransactionWithIsolatedLeader , received role change from IsolatedLeader to Leader 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received role changed for member-1-shard-cars-testTransactionWithIsolatedLeader from IsolatedLeader to Leader 09:33:30,42 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All Shards are ready - data store testTransactionWithIsolatedLeader is ready 09:33:30,43 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader Received follower initial sync status for member-2-shard-cars-testTransactionWithIsolatedLeader status sync done true 09:33:30,888 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving shard 1 connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,888 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:30,888 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:30,889 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:30,889 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:30,889 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Found primary for cars: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fa4c939] 09:33:30,889 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ActorUtils - findPrimaryShardAsync received: LocalPrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785, localShardDataTree=org.opendaylight.yangtools.yang.data.tree.impl.InMemoryDataTree@fa4c939] 09:33:30,889 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@6b2436a, attempting to connect 09:33:30,890 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, sequence=0, alternates=[], dataTree present=true, maxMessages=1000} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: resolving connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} previous state FLUSHED 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} previous state SEALED 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=OPEN} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} allocated sequence 0 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@3e6eaeb3 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=816240530438} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Replaying queued request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=OPEN} 09:33:30,890 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} allocated sequence 0 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED} enqueing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@41d9f73b 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=804229094381} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} replaying messages to old proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Allocated proxy member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} created successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} reconnected while being sealed, propagating state to successor LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=OPEN} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request ModifyTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], modifications=1} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@234c3c finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=SUCCESSOR} 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@409456eb 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b307218 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@27878b0f, splicing queue 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@1b307218 queue spliced 0 messages 09:33:30,891 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: replaced connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} in 1.291 ms 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct canCommit 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 validated 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct preCommit 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 candidate ready 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: canCommit - messages: [] 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 prepared 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 initiating direct commit 09:33:30,891 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 171 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 submitted to persistence 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 initiating direct canCommit 09:33:30,892 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,893 AM [cluster-test-shard-dispatcher-22] [WARN] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Store Tx member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1: Conflicting modification for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars. 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}]]] 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=171} 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 09:33:30,893 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 1 09:33:30,894 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 changing state from CAN_COMMIT_PENDING to FAILED 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 failed org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onFailure(FrontendReadWriteTransaction.java:441) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.failedCanCommit(SimpleShardDataTreeCohort.java:175) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$3(ShardDataTree.java:864) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:886) at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:833) at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:988) at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) at org.opendaylight.controller.cluster.datastore.TestShard.handleNonRaftCommand(TestShard.java:65) at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) at org.opendaylight.controller.cluster.datastore.TestShard.dropOrHandle(TestShard.java:83) at org.opendaylight.controller.cluster.datastore.TestShard.handleCommand(TestShard.java:76) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:270) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) Caused by: OptimisticLockFailedException{message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, errorList=[RpcError [message=Optimistic lock failed for path /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars, severity=ERROR, errorType=APPLICATION, tag=resource-denied, applicationTag=null, info=null, cause=org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction.]]} at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$3(ShardDataTree.java:848) ... 41 more Caused by: org.opendaylight.yangtools.yang.data.tree.api.ConflictingModificationAppliedException: Node was created by other transaction. at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkConflicting(SchemaAwareApplyOperation.java:93) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkWriteApplicable(SchemaAwareApplyOperation.java:203) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:130) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkChildPreconditions(AbstractNodeContainerModificationStrategy.java:411) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.checkTouchApplicable(AbstractNodeContainerModificationStrategy.java:370) at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.checkApplicable(SchemaAwareApplyOperation.java:129) at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.validate(AbstractDataTreeTip.java:37) at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$3(ShardDataTree.java:840) ... 41 more 09:33:30,894 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 0, nextIndex: 1 09:33:30,894 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:30,894 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=804229094381, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionFailure{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=0, cause=org.opendaylight.controller.cluster.access.concepts.RuntimeRequestException: CanCommit failed}} 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=171}] 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 0 09:33:30,894 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} directCommit completed 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 09:33:30,894 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: initiating purge 09:33:30,894 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,894 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} allocated sequence 1 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@2ec3c57b 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendMetadata - member-1-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@1760407}, purged=MutableUnsignedLongSet{size=0}} 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SEALED}: initiating purge 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} allocated sequence 1 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4b064fc7 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1061688783] with name member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]}, enqueuedTicks=817188265428} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=816240530438, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=0}} 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} directCommit completed 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@5260f13c 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,895 AM [main] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} allocated sequence 0 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: timer already scheduled on ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,895 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} sending request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1248/0x000000080133e630@2747fbce 09:33:30,895 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] LocalReadWriteProxyTransaction - Applying replayed request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SEALED}: initiating purge 09:33:30,895 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=817189086532} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} allocated sequence 1 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7ca76a98 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1061688783] 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]}, enqueuedTicks=817188876669} 09:33:30,895 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: allocating new ready transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,896 AM [cluster-test-akka.actor.default-dispatcher-27] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 56.97 μs 09:33:30,896 AM [cluster-test-akka.actor.default-dispatcher-23] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#1061688783] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct canCommit 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Validating transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 validated 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct preCommit 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Preparing transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 candidate ready 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - candidate rootPath: / 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] DataTreeCohortActorRegistry - Skipping unmodified candidate / 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: canCommit - messages: [] 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: preCommit - successfulFromPrevious: [] 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 prepared 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 initiating direct commit 09:33:30,896 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Starting commit for transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 69 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 2, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 1, leaderLastIndex: 2, leaderSnapShotIndex: -1 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}]]] 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 submitted to persistence 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 09:33:30,897 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, logIndex: 3, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}], SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}]]] 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 2 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 09:33:30,898 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log Simple{size=69} 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 3 09:33:30,898 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 0 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=2, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:30,898 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 2 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 0 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from -1 to 0 09:33:30,898 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 0 data Simple{size=171} 09:33:30,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendMetadata - member-2-shard-cars-testTransactionWithIsolatedLeader: Added client Enabled{identifier=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@4b5f1833}, purged=MutableUnsignedLongSet{size=0}} 09:33:30,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,899 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testTransactionWithIsolatedLeader: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#644082873] with name member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher 09:33:30,899 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader/member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher#644082873] 09:33:30,899 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:33:30,900 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 77.73 μs 09:33:30,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64}] 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 2, nextIndex: 3 09:33:30,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=2, term=4, payload=Simple{size=69}] 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 2, lastAppliedIndex: 0 09:33:30,900 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=0 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@4c00f9aa 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 2 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Resuming commit of transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] CompositeDataTreeCohort - member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1: commit - successfulFromPrevious: [] 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] SimpleShardDataTreeCohort - Transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 changing state from COMMIT_PENDING to COMMITTED 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 3, leaderSnapShotIndex: 1 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 09:33:30,900 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]}, enqueuedTicks=817188265428, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, sequence=1}} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE}: purge completed 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=DONE} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR}: purge completed 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, state=SUCCESSOR} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-4] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-1-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 34.64 μs 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=CommitLocalTransactionRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-728710922], coordinated=false}, enqueuedTicks=817189086532, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionCommitSuccess{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=0}} 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED} directCommit completed 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=FLUSHED}: initiating purge 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} allocated sequence 1 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@4c7f906a 09:33:30,901 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]}, enqueuedTicks=817194517656} 09:33:30,901 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,901 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persist data SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 09:33:30,901 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 09:33:30,901 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Replicate message: identifier: member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, logIndex: 4, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:30,902 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithIsolatedLeader, generation=1} 09:33:30,902 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}] 09:33:30,902 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=2, prevLogTerm=4, leaderCommit=2, replicatedToAllIndex=1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}]]] 09:33:30,902 AM [cluster-test-shard-dispatcher-30] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,902 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Number of entries to be appended = 1 09:33:30,902 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): After cleanup, lastIndex: 2, entries to be added from: 0 09:33:30,902 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Append entry to log PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Append log entry and persist SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Log size is now 4 09:33:30,903 AM [cluster-test-shard-dispatcher-30] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Commit index set to 2 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testTransactionWithIsolatedLeader, logLastIndex=3, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 2, context.getLastApplied(): 0, lastIndex(): 3 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 1 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 0 to 1 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1, size=64} 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-1-1 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Setting last applied to 2 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 1 to 2 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 2 data Simple{size=69} 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] ShardDataTree - member-2-shard-cars-testTransactionWithIsolatedLeader: Applying foreign transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testTransactionWithIsolatedLeader: Committed transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 1 for term 4 09:33:30,903 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testTransactionWithIsolatedLeader: publishChanges: rootPath: / 09:33:30,903 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testTransactionWithIsolatedLeader: Elapsed time for generation of change events for member-2-shard-cars-testTransactionWithIsolatedLeader-DTCL-publisher: 23.15 μs 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: persist complete SimpleReplicatedLogEntry [index=3, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63}] 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=2 09:33:30,903 AM [cluster-test-shard-dispatcher-31] [INFO] TestShard - Stopping Shard member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,904 AM [cluster-test-shard-dispatcher-31] [DEBUG] MessageAssembler - member-2-shard-cars-testTransactionWithIsolatedLeader (Follower): Closing 09:33:30,904 AM [cluster-test-shard-dispatcher-31] [DEBUG] TestShard - member-2-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:33:30,904 AM [cluster-test-akka.actor.default-dispatcher-23] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:33:30,904 AM [cluster-test-shard-dispatcher-31] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:30,904 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:30,904 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0} 09:33:30,904 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:33:30,904 AM [cluster-test-akka.actor.default-dispatcher-28] [DEBUG] MessageAssembler - member-2-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:30,904 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Issuing gracefulStop to shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:30,904 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Shutting down ShardManager shard-manager-testTransactionWithIsolatedLeader - waiting on 1 shards 09:33:30,904 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: onShutDown 09:33:30,904 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Initiating leader transfer 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@726da5f0 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@726da5f0 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Attempting to transfer leadership 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 3, leaderLastIndex: 4, leaderSnapShotIndex: 1 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 3 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testTransactionWithIsolatedLeader updated: matchIndex: 3, nextIndex: 4 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Applying to log - commitIndex: 3, lastAppliedIndex: 2 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Setting last applied to 3 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Moving last applied index from 2 to 3 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Applying state for log index 3 data PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, size=63} 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] ShardDataTree - member-1-shard-cars-testTransactionWithIsolatedLeader: replication of member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@5c5090cb 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testTransactionWithIsolatedLeader: finished purging transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, history=0, cookie=1} 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testTransactionWithIsolatedLeader: Purged transaction member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: fakeSnapshot purging log to 2 for term 4 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testTransactionWithIsolatedLeader: active: true, followerNextIndex: 4, leaderLastIndex: 4, leaderSnapShotIndex: 2 09:33:30,905 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:30,906 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:30,906 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): tryToCompleteLeadershipTransfer: followerId: member-2-shard-cars-testTransactionWithIsolatedLeader, matchIndex: 3, lastIndex: 4, isVoting: true 09:33:30,906 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Persisting ApplyJournalEntries with index=3 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-728710922]}, enqueuedTicks=817188876669, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=TransactionPurgeResponse{target=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, sequence=1}} 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE}: purge completed 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@18653ac2 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=DONE} 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractProxyTransaction - LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR}: purge completed 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@1ae46a98 purged transaction LocalReadWriteProxyTransaction{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-0-1, state=SUCCESSOR} 09:33:30,906 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderId=null, leaderPayloadVersion=12] 09:33:30,906 AM [cluster-test-shard-dispatcher-24] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ClientActorBehavior - BackendInfo for shard 1 is now stale 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 superseded by org.opendaylight.controller.cluster.access.client.SimpleReconnectForwarder@357e9655, splicing queue 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientConnection - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} scheduling timeout in 30 seconds 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@5ce44156 queue spliced 1 messages 09:33:30,906 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} reconnecting as ReconnectingClientConnection{client=ClientIdentifier{frontend=member-1-frontend-datastore-testTransactionWithIsolatedLeader, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:30,907 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] ClientActorBehavior - member-1-frontend-datastore-testTransactionWithIsolatedLeader: refreshing backend for shard 1 09:33:30,907 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Invalidating backend information ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader/member-1-shard-cars-testTransactionWithIsolatedLeader#-1632877785], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present} 09:33:30,907 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:30,907 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:30,907 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] 09:33:30,907 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:30,907 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: Scheduling 20000 ms timer to wait for shard ShardInformation [shardId=member-1-shard-cars-testTransactionWithIsolatedLeader, leaderAvailable=true, actorInitialized=true, followerSyncStatus=false, role=Leader, leaderId=null, activeMember=true] 09:33:31,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,118 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,479 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,479 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,598 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,598 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,719 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,719 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:31,959 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:31,959 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,319 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,559 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,559 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:32,919 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:32,919 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,39 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,39 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,158 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,159 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,279 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,279 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,399 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,399 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,519 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,519 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,639 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,758 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,879 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:33,898 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$LocalSingle@490e6368 aborted transaction LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} 09:33:33,898 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy LocalReadOnlyProxyTransaction{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, state=OPEN} sending request AbortLocalTransactionRequest{target=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#1281953756]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@1e107cfb 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testSingleShardTransactionsWithLeaderChanges, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test/user/shardmanager-testSingleShardTransactionsWithLeaderChanges/member-2-shard-cars-testSingleShardTransactionsWithLeaderChanges#1007633106], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=present}} 09:33:33,899 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testSingleShardTransactionsWithLeaderChanges-fe-0-txn-2-0} 09:33:33,899 AM [Cleaner-0] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@2d6969ca aborted transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SUCCESSOR} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1209/0x000000080132b558@46a58cd9 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding, generation=0}, cookie=1} has a successor, not scheduling timer 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:33:33,899 AM [Cleaner-0] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, state=SEALED} sending request ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=0, protocol=ABORT} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1325/0x000000080135d1a8@54ba1fb 09:33:33,899 AM [Cleaner-0] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@38e77b08 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=0, protocol=ABORT}, enqueuedTicks=820192580255} 09:33:33,899 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: slice: identifier: member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=3, message=ModifyTransactionRequest{target=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-1721592886], modifications=0, protocol=ABORT}} 09:33:33,899 AM [Cleaner-0] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: Message size: 294 bytes, total slices to send: 1 09:33:33,899 AM [Cleaner-0] [DEBUG] MessageSlicer - member-2-frontend-datastore-testTransactionWithShardLeaderNotResponding_slicer-id-918: Message does not need to be sliced - sending original message 09:33:33,899 AM [Cleaner-0] [INFO] ClientBackedTransaction - Aborted orphan transaction ClientSnapshot{identifier=member-2-datastore-testTransactionWithShardLeaderNotResponding-fe-0-txn-0-0} 09:33:33,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:33,999 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,119 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,239 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,359 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,480 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,480 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,598 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,599 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,719 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,719 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,839 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:34,959 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:34,959 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,79 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,199 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,319 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,439 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,559 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,559 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,678 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,798 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,799 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:35,919 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:35,919 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,38 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,158 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,159 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,279 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,279 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,399 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,399 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,518 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,518 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,639 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,639 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,878 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,878 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:36,998 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:36,998 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,358 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,358 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,479 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,479 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,599 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,599 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,719 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,719 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,838 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,839 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:37,959 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:37,959 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,79 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,79 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,199 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,199 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,318 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,318 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,439 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,439 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,559 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,559 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,679 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,679 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,798 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,798 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:38,918 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:38,919 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,39 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,39 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,159 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,159 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,279 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,279 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,398 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,398 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,519 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,519 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,639 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,639 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,759 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,879 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,879 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:39,999 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:39,999 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,119 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,239 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,359 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,359 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,478 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,478 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,599 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,599 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,719 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,719 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,839 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): sendAppendEntries: 4 is present for follower member-2-shard-cars-testTransactionWithIsolatedLeader 09:33:40,839 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Sending AppendEntries to follower member-2-shard-cars-testTransactionWithIsolatedLeader: AppendEntries [leaderId=member-1-shard-cars-testTransactionWithIsolatedLeader, prevLogIndex=3, prevLogTerm=4, leaderCommit=3, replicatedToAllIndex=2, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=4, term=4, payload=PurgeTransactionPayload{identifier=member-1-datastore-testTransactionWithIsolatedLeader-fe-0-txn-2-1, size=64}]]] 09:33:40,959 AM [cluster-test-shard-dispatcher-24] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testTransactionWithIsolatedLeader: Failed to transfer leadership in 10.05 s 09:33:40,959 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: In unpauseLeader 09:33:40,959 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: leader transfer failed - sending PoisonPill 09:33:40,959 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader): Leadership transfer expired 09:33:40,960 AM [cluster-test-shard-dispatcher-24] [INFO] TestShard - Stopping Shard member-1-shard-cars-testTransactionWithIsolatedLeader 09:33:40,960 AM [cluster-test-shard-dispatcher-24] [DEBUG] MessageSlicer - member-1-shard-cars-testTransactionWithIsolatedLeader (Leader)_slicer-id-939: Closing 09:33:40,960 AM [cluster-test-shard-dispatcher-24] [DEBUG] TestShard - member-1-shard-cars-testTransactionWithIsolatedLeader: Aborting 0 pending queued transactions 09:33:40,960 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardManager - shard-manager-testTransactionWithIsolatedLeader: All shards shutdown - sending PoisonPill to self 09:33:40,960 AM [cluster-test-shard-dispatcher-24] [INFO] ShardManager - Stopping ShardManager shard-manager-testTransactionWithIsolatedLeader 09:33:40,961 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:40,962 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testTransactionWithIsolatedLeader: shutting down 09:33:40,962 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] MessageAssembler - member-1-frontend-datastore-testTransactionWithIsolatedLeader: Closing 09:33:40,962 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testTransactionWithIsolatedLeader#762618045] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:40,962 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:40,963 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:40,967 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:40,967 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:40,968 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:40,968 AM [cluster-test-akka.actor.default-dispatcher-10] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:40,969 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:40,981 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:40,981 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:41,981 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-8085880534945855403] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:42,1 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [-2163535670106646647] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:42,3 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:42,3 AM [cluster-test-akka.actor.default-dispatcher-28] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:42,5 AM [cluster-test-akka.actor.default-dispatcher-28] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:42.007] [cluster-test-akka.actor.internal-dispatcher-3] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:42,14 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:42,14 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. [WARN] [04/25/2024 09:33:42.018] [cluster-test-akka.actor.internal-dispatcher-8] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:42,40 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Slf4jLogger - Slf4jLogger started 09:33:42,47 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2558] with UID [-6471722690519811121] 09:33:42,47 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Starting up, Akka version [2.6.21] ... 09:33:42,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Registered cluster JMX MBean [akka:type=Cluster] 09:33:42,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Started up successfully 09:33:42,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:42,49 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:42,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2558] is JOINING itself (with roles [member-1, dc-default], version [0.0.0]) and forming new cluster 09:33:42,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - is the new leader among reachable nodes (more leaders may exist) 09:33:42,72 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2558] to [Up] 09:33:42,76 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:42,84 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2559] with UID [4087651311712701120] 09:33:42,84 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Starting up, Akka version [2.6.21] ... 09:33:42,96 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:42,96 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Started up successfully 09:33:42,99 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:42,99 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:42,136 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Slf4jLogger - Slf4jLogger started 09:33:42,146 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] ArteryTransport - Remoting started with transport [Artery tcp]; listening on address [akka://cluster-test@127.0.0.1:2557] with UID [8742836011444099979] 09:33:42,146 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Starting up, Akka version [2.6.21] ... 09:33:42,148 AM [cluster-test-akka.actor.default-dispatcher-4] [WARN] Cluster - Could not register Cluster JMX MBean with name=akka:type=Cluster as it is already registered. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:42,148 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Started up successfully 09:33:42,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No downing-provider-class configured, manual cluster downing required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#downing 09:33:42,160 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - No seed-nodes configured, manual cluster join required, see https://doc.akka.io/docs/akka/current/typed/cluster.html#joining 09:33:42,161 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:42,162 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:42,163 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 09:33:42,166 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:33:42,168 AM [cluster-test-akka.actor.default-dispatcher-22] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1455570326] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,169 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2559] is JOINING, roles [member-2, dc-default], version [0.0.0] 09:33:42,171 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:33:42,171 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] RecoveringClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:42,171 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@2ba85fee to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fe3dcff 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,171 AM [cluster-test-akka.actor.default-dispatcher-5] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@1ab7a60b 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,171 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,171 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-1-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1714037622171, None)) 09:33:42,172 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] SavingClientActorBehavior - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037622170,0,0)) 09:33:42,172 AM [cluster-test-akka.actor.default-dispatcher-25] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@5fe3dcff to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@600bbac3 09:33:42,172 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 09:33:42,172 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:42,173 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2559] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:42,173 AM [main] [WARN] AbstractModuleShardConfigProvider - Reading Config from resource 09:33:42,174 AM [main] [INFO] AbstractDataStore - Creating ShardManager : shardmanager-testWriteTransactionWithMultipleShards 09:33:42,175 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@2e1aa55f 09:33:42,175 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:42,175 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,179 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1134204420] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [1] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,179 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Starting ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:33:42,181 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-1-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515] 09:33:42,181 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - Creating Shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,181 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 09:33:42,181 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,181 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1764244922] 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#424745610 created and ready for shard:member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-notifier#424745610] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#-1764244922] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Shard created : member-1-shard-people-testWriteTransactionWithMultipleShards, persistent : true 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-27] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1450634524 created and ready for shard:member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,182 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - Starting recovery for member-1-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 09:33:42,182 AM [cluster-test-akka.actor.default-dispatcher-5] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-notifier#-1450634524] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,184 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:42,184 AM [cluster-test-shard-dispatcher-28] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:42,184 AM [cluster-test-shard-dispatcher-28] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,184 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,184 AM [cluster-test-akka.actor.default-dispatcher-25] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:33:42,184 AM [cluster-test-shard-dispatcher-21] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,184 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-1-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248] 09:33:42,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 09:33:42,185 AM [cluster-test-shard-dispatcher-21] [DEBUG] ShardInformation - Shard member-1-shard-cars-testWriteTransactionWithMultipleShards is initialized 09:33:42,185 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:33:42,185 AM [cluster-test-shard-dispatcher-21] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 09:33:42,187 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1551790106] 09:33:42,187 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/shard-member-1:snapshot-read#1551790106] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:42,188 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: no initial generation override, starting from 0 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,188 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RecoveringClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: persisting new identifier ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,188 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.RecoveringClientActorBehavior@4597ff89 to org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@204a2e96 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 09:33:42,188 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: stashing command org.opendaylight.controller.cluster.databroker.actors.dds.GetClientRequest@2475be47 09:33:42,188 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Shard member-1-shard-people-testWriteTransactionWithMultipleShards is initialized 09:33:42,188 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - Recovery complete : shard-manager-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:33:42,189 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Creating local shard: member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-1}, address: akka://cluster-test@127.0.0.1:2558 09:33:42,189 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,189 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from null to Follower 09:33:42,189 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: SaveSnapshotSuccess(SnapshotMetadata(member-2-frontend-datastore-testWriteTransactionWithMultipleShards, 0, 1714037622189, None)) 09:33:42,189 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SavingClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: got command: DeleteSnapshotsSuccess(SnapshotSelectionCriteria(9223372036854775807,1714037622188,0,0)) 09:33:42,190 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: switched from org.opendaylight.controller.cluster.access.client.SavingClientActorBehavior@204a2e96 to org.opendaylight.controller.cluster.databroker.actors.dds.DistributedDataStoreClientBehavior@211371d9 09:33:42,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: onRegisterForShardAvailabilityChanges: org.opendaylight.controller.cluster.datastore.shardmanager.RegisterForShardAvailabilityChanges@44f3e917 09:33:42,190 AM [main] [DEBUG] AbstractDataStore - Distributed data store client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} started 09:33:42,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Got updated SchemaContext: # of modules 3 09:33:42,190 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-cars-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards on shard member-2-shard-cars-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#541692702] 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - Creating Shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-1-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: updated peer member-1-shard-people-testWriteTransactionWithMultipleShards on member MemberName{name=member-1} with address akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards on shard member-2-shard-people-testWriteTransactionWithMultipleShards whose actor address is Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-736536548] 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Shard created : member-2-shard-people-testWriteTransactionWithMultipleShards, persistent : true 09:33:42,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards#-736536548] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [2] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#-1593423205 created and ready for shard:member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:42,191 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-notifier#-1593423205] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [3] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,191 AM [cluster-test-akka.actor.default-dispatcher-11] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#481521038] 09:33:42,191 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Starting recovery for member-2-shard-people-testWriteTransactionWithMultipleShards with journal batch size 1 09:33:42,192 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Node [akka://cluster-test@127.0.0.1:2557] is JOINING, roles [member-3, dc-default], version [0.0.0] 09:33:42,192 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#481521038] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [4] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,192 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Shard created : member-2-shard-cars-testWriteTransactionWithMultipleShards, persistent : true 09:33:42,193 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Starting recovery for member-2-shard-cars-testWriteTransactionWithMultipleShards with journal batch size 1 09:33:42,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - Welcome from [akka://cluster-test@127.0.0.1:2558] 09:33:42,193 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RoleChangeNotifier - RoleChangeNotifier:akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-263630109 created and ready for shard:member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,193 AM [cluster-test-akka.actor.default-dispatcher-10] [DEBUG] ShardSnapshotActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1430260803] 09:33:42,194 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards#541692702] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [5] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,194 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-notifier#-263630109] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [6] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,194 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/shard-member-2:snapshot-read#1430260803] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [7] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:42,194 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:42,194 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:42,195 AM [cluster-test-shard-dispatcher-27] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,195 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [people] 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Shard member-2-shard-people-testWriteTransactionWithMultipleShards is initialized 09:33:42,195 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Peer address for peer member-1-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,195 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:33:42,195 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-people-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:33:42,195 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-people-testWriteTransactionWithMultipleShards from null to Follower 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onReceiveRecover: Received message class akka.persistence.RecoveryCompleted$ from Actor[akka://cluster-test/deadLetters] 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Recovery completed - Switching actor to Follower - last log index = -1, last log term = -1, snapshot index = -1, snapshot term = -1, journal size = 0 09:33:42,195 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Initializing shard [cars] 09:33:42,195 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardInformation - Shard member-2-shard-cars-testWriteTransactionWithMultipleShards is initialized 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [DEBUG] DataTreeChangeListenerSupport - member-2-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-2-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,195 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , received role change from null to Follower 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Peer address for peer member-1-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,195 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-2-shard-cars-testWriteTransactionWithMultipleShards , registered listener akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards 09:33:42,195 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-2-shard-cars-testWriteTransactionWithMultipleShards from null to Follower 09:33:42,240 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1455570326] 09:33:42,241 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : cars 09:33:42,241 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515] 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 09:33:42,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,418 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-cars-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,418 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:33:42,418 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Follower to Candidate 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Checking if this node is isolated in the cluster unreachable set [],all members [Member(akka://cluster-test@127.0.0.1:2557, Joining), Member(akka://cluster-test@127.0.0.1:2558, Up), Member(akka://cluster-test@127.0.0.1:2559, Joining)] self member: Member(akka://cluster-test@127.0.0.1:2558, Up) 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Received ElectionTimeout - switching to Candidate 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Election: Candidate has following voting peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=1, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 1 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower) :- Switching from behavior Follower to Candidate, election term: 1 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] MessageAssembler - member-1-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: false, hasLeader : false 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onStateChanged: Closing all transaction chains because shard member-1-shard-people-testWriteTransactionWithMultipleShards is no longer the leader 09:33:42,477 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Follower to Candidate 09:33:42,477 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Follower to Candidate 09:33:42,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:42,727 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,727 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:33:42,744 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:42,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:42,797 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=2, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:42,797 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 2 09:33:42,815 AM [cluster-test-shard-dispatcher-22] [INFO] DefaultConfigParamsImpl - Trying to use custom RaftPolicy org.opendaylight.controller.cluster.raft.policy.DisableElectionsRaftPolicy 09:33:42,987 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:42,987 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:42,987 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:33:43,57 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:43,57 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=3, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,57 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 3 09:33:43,68 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2557] to [Up] 09:33:43,68 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2558] - Leader is moving node [akka://cluster-test@127.0.0.1:2559] to [Up] 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - Peer address for peer member-2-shard-people-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,69 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Peer address for peer member-2-shard-cars-testWriteTransactionWithMultipleShards set to akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-3}, address: akka://cluster-test@127.0.0.1:2557 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-3-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2557/user/shardmanager-testWriteTransactionWithMultipleShards/member-3-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received MemberUp: memberName: MemberName{name=member-2}, address: akka://cluster-test@127.0.0.1:2559 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-cars-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [INFO] ShardInformation - updatePeerAddress for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,75 AM [cluster-test-shard-dispatcher-22] [DEBUG] ShardInformation - Sending PeerAddressResolved for peer member-2-shard-people-testWriteTransactionWithMultipleShards with address akka://cluster-test@127.0.0.1:2559/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards to akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,172 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Cluster - Cluster Node [akka://cluster-test@127.0.0.1:2557] - is the new leader among reachable nodes (more leaders may exist) 09:33:43,307 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:43,307 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,307 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:33:43,308 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-cars-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,309 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:43,310 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:43,310 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:43,311 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-cars-testWriteTransactionWithMultipleShards] 09:33:43,311 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:43,318 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards: Closing 09:33:43,318 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:43,318 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-cars-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,318 AM [cluster-test-shard-dispatcher-29] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-cars-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:33:43,318 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done false 09:33:43,318 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:33:43,318 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:43,318 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:43,318 AM [cluster-test-shard-dispatcher-22] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-cars-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:33:43,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:43,318 AM [cluster-test-shard-dispatcher-22] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:43,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:33:43,319 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-cars-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:33:43,319 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-cars-testWriteTransactionWithMultipleShards from Candidate to Leader 09:33:43,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:33:43,319 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:43,347 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Received ElectionTimeout 09:33:43,347 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,347 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Starting new election term 4 09:33:43,347 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): Sending RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] to peer member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Found higher term in RequestVote rpc, verifying whether it's safe to update term. 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Cluster state: [] 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Candidate in requestVote:RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] with higher term appears reachable, updating term. 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Term 4 in "RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1]" message is greater than follower's term 0 - updating term 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=null 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): In requestVote: RequestVote [term=4, candidateId=member-1-shard-people-testWriteTransactionWithMultipleShards, lastLogIndex=-1, lastLogTerm=-1] - currentTerm: 4, votedFor: null, lastIndex: -1, lastTerm: -1 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Set currentTerm=4, votedFor=member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,349 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): requestVote returning: RequestVoteReply [term=4, voteGranted=true] 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): RaftRPC message received RequestVoteReply [term=4, voteGranted=true], my term is 4 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate): handleRequestVoteReply: RequestVoteReply [term=4, voteGranted=true], current voteCount: 1 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Election: Leader has following peers: [member-2-shard-people-testWriteTransactionWithMultipleShards] 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: false, followerNextIndex: -1, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [INFO] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Candidate) :- Switching from behavior Candidate to Leader, election term: 4 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] DataTreeChangeListenerSupport - member-1-shard-people-testWriteTransactionWithMultipleShards: onLeadershipChange, isLeader: true, hasLeader : true 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: became leader with frontend state for [] 09:33:43,350 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RoleChangeNotifier - RoleChangeNotifier for member-1-shard-people-testWriteTransactionWithMultipleShards , received role change from Candidate to Leader 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:33:43,350 AM [cluster-test-shard-dispatcher-32] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:33:43,351 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received role changed for member-1-shard-people-testWriteTransactionWithMultipleShards from Candidate to Leader 09:33:43,351 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 09:33:43,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards: Closing 09:33:43,352 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: getLeaderAddress leaderId = member-1-shard-people-testWriteTransactionWithMultipleShards peerAddress = akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,352 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-2-shard-people-testWriteTransactionWithMultipleShards, leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderPayloadVersion=12] 09:33:43,352 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:33:43,352 AM [cluster-test-shard-dispatcher-27] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:33:43,352 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All Shards are ready - data store testWriteTransactionWithMultipleShards is ready 09:33:43,352 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done false 09:33:43,352 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: -1, nextIndex: 0 09:33:43,353 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: -1, leaderSnapShotIndex: -1 09:33:43,397 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindLocalShard to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1455570326] 09:33:43,398 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: findLocalShard : people 09:33:43,398 AM [main] [DEBUG] ActorUtils - Local shard found Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248] 09:33:43,398 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1e49f8ba: creating a new transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:33:43,398 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 1 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:33:43,398 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 1 to shard cars 09:33:43,398 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 1 to shard cars 09:33:43,398 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1134204420] 09:33:43,398 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=cars, waitUntilReady=true] 09:33:43,398 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 1 09:33:43,398 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:33:43,398 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for cars: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:33:43,398 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:43,399 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:33:43,399 AM [main] [DEBUG] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving shard 2 connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:33:43,399 AM [main] [DEBUG] ModuleShardBackendResolver - Resolving cookie 2 to shard people 09:33:43,399 AM [main] [DEBUG] AbstractShardBackendResolver - Resolving cookie 2 to shard people 09:33:43,399 AM [main] [DEBUG] ActorUtils - Sending message class org.opendaylight.controller.cluster.datastore.messages.FindPrimary to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1134204420] 09:33:43,399 AM [main] [DEBUG] AbstractClientHistory - Created proxyId LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} for history LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} shard 2 09:33:43,399 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard cars resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@4b778e3b, attempting to connect 09:33:43,399 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:33:43,399 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:33:43,399 AM [main] [DEBUG] AbstractClientHistory - Local history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1e49f8ba readied transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-0 09:33:43,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: In findPrimary: org.opendaylight.controller.cluster.datastore.messages.FindPrimary [shardName=people, waitUntilReady=true] 09:33:43,399 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@6ee883c 09:33:43,399 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Found primary for people: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:33:43,399 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} scheduling timeout in 30 seconds 09:33:43,399 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ActorUtils - findPrimaryShardAsync received: RemotePrimaryShardFound [primaryPath=akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards, primaryVersion=12] 09:33:43,399 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=FLUSHED} sending request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1233/0x00000008013332d0@12ed0b67 09:33:43,399 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} scheduling timeout in 30 seconds 09:33:43,399 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Shard people resolved to org.opendaylight.controller.cluster.datastore.messages.PrimaryShardInfo@54617beb, attempting to connect 09:33:43,400 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 09:33:43,401 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=13081, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:43,401 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} is not yet known 09:33:43,401 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: created state Enabled{clientId=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, nanosAgo=25190, purgedHistories=MutableUnsignedLongSet{size=0}} for client ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:43,402 AM [ForkJoinPool.commonPool-worker-4] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:33:43,402 AM [ForkJoinPool.commonPool-worker-2] [DEBUG] AbstractShardBackendResolver - Resolved backend information to ConnectClientSuccess{target=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, sequence=0, alternates=[], dataTree present=false, maxMessages=1000} 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 1 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent} 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4cbc0d2d instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b20b5e2 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} previous state FLUSHED 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@5b20b5e2 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,402 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=OPEN} allocated sequence 0 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@30237220 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} scheduling timeout in 30 seconds 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6ace3be5 transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=829692777563} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, message: Envelope{sessionId=0, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: Message size: 391 bytes, total slices to send: 1 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: Message does not need to be sliced - sending original message 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1e49f8ba finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@53180ef0 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6cc0e461 superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@3adee5a4, splicing queue 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@6cc0e461 queue spliced 0 messages 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} in 1.158 ms 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolved shard 2 to ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent} 09:33:43,403 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: resolving connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - History org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c9286a5 instantiated successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378bb881 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Start reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} previous state FLUSHED 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} replaying messages to old proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} towards successor org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378bb881 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} created successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Replaying queued request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} to successor RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=OPEN} allocated sequence 0 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} enqueing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$2295/0x00000008016fbd68@11bd8c1c 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: connection ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} scheduling timeout in 30 seconds 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@fbfd78f transmitting entry ConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=829692881545} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, message: Envelope{sessionId=1, txSequence=0, message=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] SlicedMessageState - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: Message size: 396 bytes, total slices to send: 1 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] MessageSlicer - member-2-frontend-datastore-testWriteTransactionWithMultipleShards_slicer-id-943: Message does not need to be sliced - sending original message 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientHistory - Client history org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1e49f8ba finishing reconnect to ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Finishing reconnect of proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Finished reconnecting proxy history org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$ReconnectCohort@6496bf74 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@678581ee superseded by org.opendaylight.controller.cluster.databroker.actors.dds.BouncingReconnectForwarder@71c60eca, splicing queue 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Connection org.opendaylight.controller.cluster.access.client.TransmitQueue$Halted@678581ee queue spliced 0 messages 09:33:43,404 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] ClientActorBehavior - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: replaced connection ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} with ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} in 693.2 μs 09:33:43,404 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 ready 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 to ready 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating canCommit 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 validated 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,405 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed canCommit 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: readying transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 ready 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: transitioned member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 to ready 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating canCommit 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Validating transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 validated 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from CAN_COMMIT_PENDING to CAN_COMMIT_COMPLETE 09:33:43,405 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed canCommit 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=829692777563, sessionId=0, txSequence=0} with Envelope{sessionId=0, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=0}} 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} canCommit completed 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ModifyTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], modifications=1, protocol=THREE_PHASE}, enqueuedTicks=829692881545, sessionId=1, txSequence=0} with Envelope{sessionId=1, txSequence=0, message=TransactionCanCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=0}} 09:33:43,406 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} canCommit completed 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 1 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@73eb1edb 09:33:43,406 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 1 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@1b6d7f2 09:33:43,406 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:43,406 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6ace3be5 transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829700176101} 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 1 09:33:43,406 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1216/0x000000080132f3b0@7b7a4290 09:33:43,406 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:33:43,407 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 1 09:33:43,407 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1326/0x000000080135d3e0@71339646 09:33:43,407 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:33:43,407 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@fbfd78f transmitting entry ConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829700340184} 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating preCommit 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 candidate ready 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - candidate rootPath: / 09:33:43,407 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating preCommit 09:33:43,407 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Preparing transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: canCommit - messages: [] 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: preCommit - successfulFromPrevious: [] 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 prepared 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:43,407 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 completed preCommit 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 candidate ready 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - candidate rootPath: / 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: canCommit - messages: [] 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: preCommit - successfulFromPrevious: [] 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 prepared 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from PRE_COMMIT_PENDING to PRE_COMMIT_COMPLETE 09:33:43,408 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed preCommit 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829700176101, sessionId=0, txSequence=1} with Envelope{sessionId=0, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=1}} 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPreCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829700340184, sessionId=1, txSequence=1} with Envelope{sessionId=1, txSequence=1, message=TransactionPreCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=1}} 09:33:43,409 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} preCommit completed, clearing successfulRequests 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 2 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6d3b0688 09:33:43,409 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} allocated sequence 2 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@6d3b0688 09:33:43,409 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:43,409 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6ace3be5 transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829702816067} 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 2 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@447cb77d 09:33:43,409 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} allocated sequence 2 09:33:43,409 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED} sending request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1239/0x0000000801337818@447cb77d 09:33:43,409 AM [main] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:33:43,409 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@fbfd78f transmitting entry ConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829702972269} 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendReadWriteTransaction - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 initiating commit 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendReadWriteTransaction - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 initiating commit 09:33:43,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Starting commit for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 180 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:43,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] CommitTransactionPayload - Initial buffer capacity 512, actual serialized size 176 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:43,410 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 09:33:43,410 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 09:33:43,411 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 0, payload: class org.opendaylight.controller.cluster.datastore.persisted.CommitTransactionPayload$Simple, isSendImmediate: true 09:33:43,411 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 submitted to persistence 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 0, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 0 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 submitted to persistence 09:33:43,411 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:33:43,411 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}]]] 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=180} 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=-1, prevLogTerm=-1, leaderCommit=-1, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}]]] 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: -1, entries to be added from: 0 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Append entry to log Simple{size=176} 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 09:33:43,412 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Log size is now 1 09:33:43,412 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-cars-testWriteTransactionWithMultipleShards, logLastIndex=0, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:43,413 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=176}] 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:33:43,413 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=0, term=4, payload=Simple{size=180}] 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:33:43,413 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-cars-testWriteTransactionWithMultipleShards status sync done true 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:33:43,413 AM [cluster-test-shard-dispatcher-27] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards Received follower initial sync status for member-2-shard-people-testWriteTransactionWithMultipleShards status sync done true 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-cars-testWriteTransactionWithMultipleShards updated: matchIndex: 0, nextIndex: 1 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 0, lastAppliedIndex: -1 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 0 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendMetadata - member-1-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@567a4285}, purged=MutableUnsignedLongSet{size=0}} 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Resuming commit of transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2: commit - successfulFromPrevious: [] 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 changing state from COMMIT_PENDING to COMMITTED 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendMetadata - member-1-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@22ec3c54}, purged=MutableUnsignedLongSet{size=0}} 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] CompositeDataTreeCohort - member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1: commit - successfulFromPrevious: [] 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] SimpleShardDataTreeCohort - Transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 changing state from COMMIT_PENDING to COMMITTED 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2038572058] with name member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-1-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1969428175] with name member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 09:33:43,413 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1969428175] 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:43,413 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 0, leaderSnapShotIndex: -1 09:33:43,413 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:33:43,413 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 70.07 μs 09:33:43,413 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:33:43,414 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1969428175] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,414 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2038572058] 09:33:43,414 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-1-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:33:43,414 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] ShardDataTreeChangePublisherActor - member-1-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 53.30 μs 09:33:43,414 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#2038572058] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829702972269, sessionId=1, txSequence=2} with Envelope{sessionId=1, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=2}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} doCommit completed 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c9286a5 completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: initiating purge 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} allocated sequence 3 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@2eb4bdb2 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SEALED}: initiating purge 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} allocated sequence 3 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7f88da0f 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=2, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-people-testWriteTransactionWithMultipleShards#118161248], sessionId=1, version=CHLORINE_SR2, maxMessages=1000, cookie=2, shard=people, dataTree=absent}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@fbfd78f transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829708441644} 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionDoCommitRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829702816067, sessionId=0, txSequence=2} with Envelope{sessionId=0, txSequence=2, message=TransactionCommitSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=2}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} doCommit completed 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@4cbc0d2d completing transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR}: initiating purge 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=180} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} allocated sequence 3 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SUCCESSOR} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@753846fc 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectingClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=SEALED}: initiating purge 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} allocated sequence 3 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTree - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, state=DONE} enqueing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} callback org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction$$Lambda$1226/0x0000000801331338@7e6a3cc4 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: timer already scheduled on ConnectedClientConnection{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, cookie=1, backend=ShardBackendInfo{actor=Actor[akka://cluster-test@127.0.0.1:2558/user/shardmanager-testWriteTransactionWithMultipleShards/member-1-shard-cars-testWriteTransactionWithMultipleShards#-1765892515], sessionId=0, version=CHLORINE_SR2, maxMessages=1000, cookie=1, shard=cars, dataTree=absent}} 09:33:43,415 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6ace3be5 transmitting entry ConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829708698849} 09:33:43,415 AM [main] [DEBUG] SingleClientHistory - org.opendaylight.controller.cluster.databroker.actors.dds.SingleClientHistory@1e49f8ba: creating a new snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Commit index set to 0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 0, context.getLastApplied(): -1, lastIndex(): 0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Moving last applied index from -1 to 0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying state for log index 0 data Simple{size=176} 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTree - member-2-shard-cars-testWriteTransactionWithMultipleShards: Applying foreign transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,415 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:33:43,415 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} allocated sequence 0 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendMetadata - member-2-shard-people-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@5dd766c9}, purged=MutableUnsignedLongSet{size=0}} 09:33:43,415 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@4d226fad 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,415 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@6ace3be5 transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=829709118596} 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-people-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-248461180] with name member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendMetadata - member-2-shard-cars-testWriteTransactionWithMultipleShards: Added client Enabled{identifier=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, current={LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0}=org.opendaylight.controller.cluster.datastore.FrontendHistoryMetadataBuilder@799538c6}, purged=MutableUnsignedLongSet{size=0}} 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=1} 09:33:43,415 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:33:43,415 AM [cluster-test-shard-dispatcher-27] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-cars-testWriteTransactionWithMultipleShards: Committed transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] ShardDataTreeChangeListenerPublisherActorProxy - member-2-shard-cars-testWriteTransactionWithMultipleShards: Created publisher actor Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1506300577] with name member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher 09:33:43,416 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=0 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1506300577] 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-cars-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - Actor created Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-248461180] 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-cars-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher: 28.15 μs 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-24] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards/member-2-shard-cars-testWriteTransactionWithMultipleShards-DTCL-publisher#-1506300577] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [8] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] DefaultShardDataTreeChangeListenerPublisher - member-2-shard-people-testWriteTransactionWithMultipleShards: publishChanges: rootPath: / 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardDataTreeChangePublisherActor - member-2-shard-people-testWriteTransactionWithMultipleShards: Elapsed time for generation of change events for member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher: 150.0 μs 09:33:43,416 AM [cluster-test-akka.actor.default-dispatcher-4] [INFO] EmptyLocalActorRef - Message [org.opendaylight.controller.cluster.common.actor.Monitor] from Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards/member-2-shard-people-testWriteTransactionWithMultipleShards-DTCL-publisher#-248461180] to Actor[akka://cluster-test/user/termination-monitor] was not delivered. [9] dead letters encountered. If this is not an expected behavior then Actor[akka://cluster-test/user/termination-monitor] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1 09:33:43,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Persist data SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 09:33:43,416 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-people-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 09:33:43,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Replicate message: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, logIndex: 1, payload: class org.opendaylight.controller.cluster.datastore.persisted.PurgeTransactionPayload, isSendImmediate: false 09:33:43,416 AM [cluster-test-shard-dispatcher-29] [DEBUG] AbstractFrontendHistory - member-1-shard-cars-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1 09:33:43,417 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-941: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, message: Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:33:43,417 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] SlicedMessageState - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-941: Message size: 332 bytes, total slices to send: 1 09:33:43,417 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards_slicer-id-941: Message does not need to be sliced - sending original message 09:33:43,417 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 09:33:43,417 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}] 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries: AppendEntries [leaderId=member-1-shard-people-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}]]] 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Number of entries to be appended = 1 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): After cleanup, lastIndex: 0, entries to be added from: 0 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Append entry to log PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Append log entry and persist SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Log size is now 2 09:33:43,418 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:33:43,418 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars}, enqueuedTicks=829709118596, sessionId=0, txSequence=4} with Envelope{sessionId=0, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0}} 09:33:43,418 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:cars?revision=2014-03-13)cars completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-1, sequence=0} 09:33:43,418 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): handleAppendEntries returning : AppendEntriesReply [term=4, success=true, followerId=member-2-shard-people-testWriteTransactionWithMultipleShards, logLastIndex=1, logLastTerm=4, forceInstallSnapshot=false, needsLeaderAddress=false, payloadVersion=12, raftVersion=5, recipientRaftVersion=5] 09:33:43,418 AM [main] [DEBUG] ProxyHistory - Allocated proxy member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 for transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-0 09:33:43,418 AM [main] [DEBUG] AbstractProxyTransaction - Transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} allocated sequence 0 09:33:43,418 AM [main] [DEBUG] AbstractProxyTransaction - Transaction proxy RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, state=OPEN} sending request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} callback org.opendaylight.controller.cluster.databroker.actors.dds.RemoteProxyTransaction$$Lambda$1258/0x00000008013415d0@591414e2 09:33:43,418 AM [main] [DEBUG] TransmitQueue - Queue org.opendaylight.controller.cluster.access.client.TransmitQueue$Transmitting@fbfd78f transmitting entry ConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=829711866813} 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): handleAppendEntriesReply - FollowerLogInformation for member-2-shard-people-testWriteTransactionWithMultipleShards updated: matchIndex: 1, nextIndex: 2 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Applying to log - commitIndex: 1, lastAppliedIndex: 0 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Setting last applied to 1 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: replication of member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 completed, invoking org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory$$Lambda$2160/0x0000000801695c90@6ceceeb9 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: finished purging transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] FrontendClientMetadataBuilder - member-1-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:33:43,419 AM [cluster-test-shard-dispatcher-32] [DEBUG] AbstractFrontendHistory - member-1-shard-people-testWriteTransactionWithMultipleShards: allocating new open snapshot member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2 09:33:43,419 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-942: slice: identifier: member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, message: Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:33:43,419 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] SlicedMessageState - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-942: Message size: 338 bytes, total slices to send: 1 09:33:43,419 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: persist complete SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68}] 09:33:43,419 AM [cluster-test-akka.actor.default-dispatcher-31] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards_slicer-id-942: Message does not need to be sliced - sending original message 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=TransactionPurgeRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3, replyTo=Actor[akka://cluster-test/user/$a#-284238120]}, enqueuedTicks=829708441644, sessionId=1, txSequence=3} with Envelope{sessionId=1, txSequence=3, message=TransactionPurgeResponse{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, sequence=3}} 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE}: purge completed 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@378bb881 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=DONE} 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Commit index set to 1 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractProxyTransaction - RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR}: purge completed 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] ProxyHistory - Proxy org.opendaylight.controller.cluster.databroker.actors.dds.ProxyHistory$RemoteSingle@1c9286a5 purged transaction RemoteProxyTransaction{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, state=SUCCESSOR} 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): applyLogToStateMachine, appendEntries.getLeaderCommit(): 1, context.getLastApplied(): 0, lastIndex(): 1 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Setting last applied to 1 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Moving last applied index from 0 to 1 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] TransmitQueue - Completing request ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Applying state for log index 1 data PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2, size=68} 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] AbstractClientConnection - Completing TransmittedConnectionEntry{request=ReadTransactionRequest{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0, replyTo=Actor[akka://cluster-test/user/$a#-284238120], snapshotOnly=true, path=/(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people}, enqueuedTicks=829711866813, sessionId=1, txSequence=4} with Envelope{sessionId=1, txSequence=4, message=ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0}} 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: looking up LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=0} instead of LocalHistoryIdentifier{client=ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0}, history=0, cookie=2} 09:33:43,420 AM [cluster-test-akka.actor.default-dispatcher-24] [DEBUG] RemoteProxyTransaction - Read request for /(urn:opendaylight:params:xml:ns:yang:controller:md:sal:dom:store:test:people?revision=2014-03-13)people completed with ReadTransactionSuccess{target=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-1-2, sequence=0} 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] FrontendClientMetadataBuilder - member-2-shard-people-testWriteTransactionWithMultipleShards: Purged transaction member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-2 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: fakeSnapshot purging log to 0 for term 4 09:33:43,420 AM [cluster-test-shard-dispatcher-28] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Persisting ApplyJournalEntries with index=1 09:33:43,420 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-2-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:43,421 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,421 AM [cluster-test-shard-dispatcher-28] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,421 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 09:33:43,421 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 09:33:43,421 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: onShutDown 09:33:43,421 AM [cluster-test-shard-dispatcher-27] [INFO] TestShard - Stopping Shard member-2-shard-people-testWriteTransactionWithMultipleShards 09:33:43,421 AM [cluster-test-shard-dispatcher-27] [DEBUG] MessageAssembler - member-2-shard-people-testWriteTransactionWithMultipleShards (Follower): Closing 09:33:43,421 AM [cluster-test-shard-dispatcher-22] [INFO] TestShard - Stopping Shard member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,421 AM [cluster-test-shard-dispatcher-22] [DEBUG] MessageAssembler - member-2-shard-cars-testWriteTransactionWithMultipleShards (Follower): Closing 09:33:43,421 AM [cluster-test-shard-dispatcher-22] [DEBUG] TestShard - member-2-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:33:43,421 AM [cluster-test-shard-dispatcher-27] [DEBUG] TestShard - member-2-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:33:43,422 AM [cluster-test-akka.actor.default-dispatcher-29] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:33:43,422 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:33:43,422 AM [main] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:33:43,422 AM [main] [INFO] AbstractDataStore - Closing data store ClientIdentifier{frontend=member-1-frontend-datastore-testWriteTransactionWithMultipleShards, generation=0} 09:33:43,422 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 09:33:43,423 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-2-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Issuing gracefulStop to shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [INFO] ShardManager - Shutting down ShardManager shard-manager-testWriteTransactionWithMultipleShards - waiting on 2 shards 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: onShutDown 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Initiating leader transfer 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@59c33d5e 09:33:43,423 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.databroker.actors.dds.AbstractDataStoreClientBehavior$$Lambda$1253/0x0000000801340210] to Actor[akka://cluster-test/user/$a#-284238120] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/$a#-284238120] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: onShutDown 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] ShardDataTree - member-1-shard-cars-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@59c33d5e 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Initiating leader transfer 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-cars-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 1, leaderLastIndex: 1, leaderSnapShotIndex: -1 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,423 AM [cluster-test-shard-dispatcher-32] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In pauseLeader, operation: org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78214d3c 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-people-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for people 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for people 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [INFO] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: Received LeaderStateChanged message: LeaderStateChanged [memberId=member-1-shard-cars-testWriteTransactionWithMultipleShards, leaderId=null, leaderPayloadVersion=12] 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - onShardAvailabilityChange for cars 09:33:43,423 AM [cluster-test-shard-dispatcher-28] [DEBUG] ModuleShardBackendResolver - No shard cookie found for cars 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] ShardDataTree - member-1-shard-people-testWriteTransactionWithMultipleShards: Pending transactions complete - running operation org.opendaylight.controller.cluster.raft.RaftActorLeadershipTransferCohort$1@78214d3c 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Attempting to transfer leadership 09:33:43,423 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Checking sendAppendEntries for follower member-2-shard-people-testWriteTransactionWithMultipleShards: active: true, followerNextIndex: 2, leaderLastIndex: 1, leaderSnapShotIndex: 0 09:33:43,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): sendAppendEntries: 1 is present for follower member-2-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,557 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Sending AppendEntries to follower member-2-shard-cars-testWriteTransactionWithMultipleShards: AppendEntries [leaderId=member-1-shard-cars-testWriteTransactionWithMultipleShards, prevLogIndex=0, prevLogTerm=4, leaderCommit=0, replicatedToAllIndex=-1, payloadVersion=12, recipientRaftVersion=5, leaderRaftVersion=5, leaderAddress=null, entries=[SimpleReplicatedLogEntry [index=1, term=4, payload=PurgeTransactionPayload{identifier=member-2-datastore-testWriteTransactionWithMultipleShards-fe-0-txn-0-1, size=68}]]] 09:33:43,677 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-cars-testWriteTransactionWithMultipleShards: Failed to transfer leadership in 254.3 ms 09:33:43,677 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: In unpauseLeader 09:33:43,678 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:33:43,678 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:33:43,678 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-cars-testWriteTransactionWithMultipleShards 09:33:43,678 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-cars-testWriteTransactionWithMultipleShards (Leader)_slicer-id-946: Closing 09:33:43,678 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-cars-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [WARN] RaftActorLeadershipTransferCohort - member-1-shard-people-testWriteTransactionWithMultipleShards: Failed to transfer leadership in 284.4 ms 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: In unpauseLeader 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: leader transfer failed - sending PoisonPill 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader): Leadership transfer expired 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [INFO] TestShard - Stopping Shard member-1-shard-people-testWriteTransactionWithMultipleShards 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] MessageSlicer - member-1-shard-people-testWriteTransactionWithMultipleShards (Leader)_slicer-id-947: Closing 09:33:43,708 AM [cluster-test-shard-dispatcher-29] [DEBUG] TestShard - member-1-shard-people-testWriteTransactionWithMultipleShards: Aborting 0 pending queued transactions 09:33:43,709 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] ShardManager - shard-manager-testWriteTransactionWithMultipleShards: All shards shutdown - sending PoisonPill to self 09:33:43,709 AM [cluster-test-shard-dispatcher-29] [INFO] ShardManager - Stopping ShardManager shard-manager-testWriteTransactionWithMultipleShards 09:33:43,709 AM [main] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:33:43,709 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] AbstractClientActor - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: shutting down 09:33:43,709 AM [cluster-test-akka.actor.default-dispatcher-30] [DEBUG] MessageAssembler - member-1-frontend-datastore-testWriteTransactionWithMultipleShards: Closing 09:33:43,710 AM [cluster-test-akka.actor.default-dispatcher-30] [INFO] RepointableActorRef - Message [org.opendaylight.controller.cluster.common.actor.ExecuteInSelfMessage] to Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1455570326] was not delivered. [10] dead letters encountered, no more dead letters will be logged in next [5.000 min]. If this is not an expected behavior then Actor[akka://cluster-test/user/shardmanager-testWriteTransactionWithMultipleShards#1455570326] may have terminated unexpectedly. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 09:33:43,718 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:43,718 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:43,722 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:43,722 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2559], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:43,722 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:43,722 AM [cluster-test-akka.actor.default-dispatcher-31] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:43,724 AM [cluster-test-akka.actor.default-dispatcher-31] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. 09:33:43,743 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:43,743 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:44,732 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2558] having UID [-6471722690519811121] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:44,757 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], message stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:44,757 AM [cluster-test-akka.actor.default-dispatcher-11] [WARN] Materializer - [outbound connection to [akka://cluster-test@127.0.0.1:2557], control stream] Upstream failed, cause: StreamTcpException: The connection has been aborted 09:33:44,760 AM [cluster-test-akka.actor.default-dispatcher-11] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:44.762] [cluster-test-akka.actor.internal-dispatcher-6] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config 09:33:44,763 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] Association - Association to [akka://cluster-test@127.0.0.1:2559] having UID [4087651311712701120] has been stopped. All messages to this UID will be delivered to dead letters. Reason: ActorSystem terminated 09:33:44,776 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Shutting down remote daemon. 09:33:44,776 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remote daemon shut down; proceeding with flushing remote transports. 09:33:44,779 AM [cluster-test-akka.actor.default-dispatcher-10] [INFO] RemoteActorRefProvider$RemotingTerminator - Remoting shut down. [WARN] [04/25/2024 09:33:44.780] [cluster-test-akka.actor.internal-dispatcher-7] [Cluster(akka://cluster-test)] Could not unregister Cluster JMX MBean with name=akka:type=Cluster as it was not found. If you are running multiple clusters in the same JVM, set 'akka.cluster.jmx.multi-mbeans-in-same-jvm = on' in config